To carry out an effective study on Pneumonia detection, one must understand about the disease, what causes it and what are the technical jargons used in this filed of study. In this section, we’ll answer in brief about these topics.
Pneumonia is an infection that inflames the air sacs in one or both lungs. The air sacs may fill with fluid or pus (purulent material), causing cough with phlegm or pus, fever, chills, and difficulty breathing. A variety of organisms, including bacteria, viruses and fungi, can cause pneumonia.
The signs and symptoms of pneumonia vary from mild to severe, depending on factors such as the type of germ causing the infection, and your age and overall health. Mild signs and symptoms often are similar to those of a cold or flu, but they last longer.
Signs and symptoms of pneumonia may include:
Blood tests: Blood tests are used to confirm an infection and to try to identify the type of organism causing the infection.
Chest X-ray: This helps your doctor diagnose pneumonia and determine the extent and location of the infection.
Pulse oximetry: This measures the oxygen level in your blood. Pneumonia can prevent your lungs from moving enough oxygen into your bloodstream.
Sputum test: A sample of fluid from your lungs (sputum) is taken after a deep cough and analyzed to help pinpoint the cause of the infection.
In our study, we will focus our research on how Pneumonia is detected with the help if chest X-rays. In below image, we can see how pneumonia is read by the doctors or radiologists in x-rays.
"Opacity refers to any area that preferentially attenuates the x-ray beam and therefore appears more opaque than the surrounding area. It is a nonspecific term that does not indicate the size or pathologic nature of the abnormality."
Any area in the chest radiograph that is whiter than it should be. If you compare the images of Sample Patient 1 and Sample Patient 2 you can see that the lower boundry of the lungs of patient 2 is obscured by opacities. In the image of Sample Patient 1 you can see the clear difference between the black lungs and the tissue below it, and in the image of Sample Patient 2 there is just this fuzziness.
'Pneumonia' is an infection in the lung, which requires review of a chest radiograph by highly trained specialists. Pneumonia shows up in a chest radiograph as an area of opacity. However, diagnosing it can be complicated and much time and effort is spent by specialists in reviewing X-rays. Chest radiograph is the most common performed diagnostic imaging study. Due to the high volume of chest radiography, it is very time consuming and intensive for the radiologists to review each image manually.
As such, an automated solution is ideal to locate the position of inflammation in an image. By having such an automated pneumonia screening system, this can assist physicians to make better clinical decisions or even replace human judgement in this area.
The link for the dataset and more information can be found here - https://www.kaggle.com/c/rsna-pneumonia-detection-challenge
The data is given in a zip file “rsna-pneumonia-detection-challenge.zip”, which contains the following items:
stage_2_train_labels.csv:
This file contains the corresponding patientID images to the folder 'stage_2_train_images' and contains the bounding box of areas of pneumonia detected in each image along with a target label of 0 or 1 for pneumonia detected.
The stage_2_train_labels dataset have 30,227 rows and 6 columns.
stage_2_detailed_class_info.csv:
This file contains the corresponding patientID along with the target class labels of the images.
The stage_2_detailed_class has 26,684 training images present.
stage_2_test_images:
This folder contains all the test dataset chest radiograph DICOM images.
stage_2_sample_submission.csv:
This file contains the corresponding patientID images to the folder 'stage_2_test_images'.
# Import TensorFlow
import tensorflow as tf
print('tensorflow version : {0}'.format(tf.__version__))
import random
random.seed(0)
tensorflow version : 2.8.0
from google.colab import drive
drive.mount('/content/drive')
Mounted at /content/drive
# Install pydicom
!pip install pydicom
Collecting pydicom
Downloading pydicom-2.3.0-py3-none-any.whl (2.0 MB)
|████████████████████████████████| 2.0 MB 4.2 MB/s
Installing collected packages: pydicom
Successfully installed pydicom-2.3.0
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
from matplotlib.patches import Rectangle
import cv2
import os
import pydicom as dicom
import seaborn as sns
sns.set_style("darkgrid")
from tqdm import tqdm, tqdm_notebook
from glob import glob
from skimage.transform import resize
from skimage import io, measure
import warnings
warnings.filterwarnings('ignore')
# Unzipping the dowloaded dataset
from zipfile import ZipFile
os.chdir('/content')
zip_name = '/content/drive/MyDrive/Colab Notebooks/rsna-pneumonia-detection-challenge.zip'
with ZipFile(zip_name, 'r') as zip:
print('Extracting all the files...')
zip.extractall()
print('Done!')
Extracting all the files... Done!
The dataset train_labels contain details about the patient-id and the co-ordinates of the bounding boxes along with the width and height.
It also contains the Binary classification column as Target which indicate whether the sample image has traces of pneumonia or not.
train_labels = pd.read_csv('/content/stage_2_train_labels.csv')
print('First five rows of Training set :')
train_labels.head(5)
First five rows of Training set :
| patientId | x | y | width | height | Target | |
|---|---|---|---|---|---|---|
| 0 | 0004cfab-14fd-4e49-80ba-63a80b6bddd6 | NaN | NaN | NaN | NaN | 0 |
| 1 | 00313ee0-9eaa-42f4-b0ab-c148ed3241cd | NaN | NaN | NaN | NaN | 0 |
| 2 | 00322d4d-1c29-4943-afc9-b6754be640eb | NaN | NaN | NaN | NaN | 0 |
| 3 | 003d8fa0-6bf1-40ed-b54c-ac657f8495c5 | NaN | NaN | NaN | NaN | 0 |
| 4 | 00436515-870c-4b36-a041-de91049b9ab4 | 264.0 | 152.0 | 213.0 | 379.0 | 1 |
# Number of entries in Train labels dataframe
print('The train_labels dataframe has {0} rows and {1} columns.'.format(train_labels.shape[0], train_labels.shape[1]))
The train_labels dataframe has 30227 rows and 6 columns.
# Number of duplicates in patientId
print('Number of unique patientId are : {0}'.format(train_labels['patientId'].nunique()))
Number of unique patientId are : 26684
Thus, the dataset contains information about 26684 patients. Out of these 26684 patients, some of them have multiple entries in the dataset.
print('No of entries which has Pneumonia : {0} ({1}%)'.format(train_labels[train_labels.Target == 1].shape[0], round(train_labels[train_labels.Target == 1].shape[0]/train_labels.shape[0]*100, 0)))
print('No of entries which don\'t have Pneumonia : {0} ({1}%)\n'.format(train_labels[train_labels.Target == 0].shape[0], round(train_labels[train_labels.Target == 0].shape[0]/train_labels.shape[0]*100, 0)))
plt.figure(figsize=(10,6))
textprops = {"fontsize":13} # Font size of text in pie chart
train_labels['Target'].value_counts().plot(kind = 'pie', autopct = '%.0f%%', explode = [0.04,0.01], labels = ['Negative', 'Positive'], startangle=0, pctdistance=0.5, labeldistance=1.1, textprops = textprops)
plt.title('Pneumonia distribution', fontdict=textprops)
plt.xlabel('')
plt.ylabel('')
plt.show()
No of entries which has Pneumonia : 9555 (32.0%) No of entries which don't have Pneumonia : 20672 (68.0%)
The dataset contains data 30,277 records for 26,684 unique patients. Suggesting some patients have more than one entry (more than one bounding boxes detecting pneumonia).
Out of 30,277 entries in the dataset,
No. of positive cases = 9,555 ~ 32%
No. of negative cases = 20,672 ~ 68%
To cross verify, we can count the number of bounding boxes with null values for x, y, width and height.
# Checking nulls in bounding box columns
print('Number of nulls in bounding box columns : {0}'.format(train_labels[['x', 'y', 'width', 'height']].isnull().sum().to_dict()))
Number of nulls in bounding box columns : {'x': 20672, 'y': 20672, 'width': 20672, 'height': 20672}
bounding_box = train_labels.groupby('patientId').size().to_frame('number_of_boxes').reset_index()
train_labels = train_labels.merge(bounding_box, on = 'patientId', how = 'left')
print('Number of patientIds per bounding box in the dataset : ')
(bounding_box.groupby('number_of_boxes').size().to_frame('number_of_patientId').reset_index().set_index('number_of_boxes').sort_values(by = 'number_of_boxes'))
Number of patientIds per bounding box in the dataset :
| number_of_patientId | |
|---|---|
| number_of_boxes | |
| 1 | 23286 |
| 2 | 3266 |
| 3 | 119 |
| 4 | 13 |
Thus, there are 23286 unique patients which have only one entry in the dataset. It also has the patientsbounding box, 3266 with 2 bounding box, 119 with 3 bounding box and 13 with 4 bounding box coordinates.
The data is divided into 3 labels.
class_labels = pd.read_csv('/content/stage_2_detailed_class_info.csv')
print('First five rows of Class labels dataset are :')
class_labels.head(5)
First five rows of Class labels dataset are :
| patientId | class | |
|---|---|---|
| 0 | 0004cfab-14fd-4e49-80ba-63a80b6bddd6 | No Lung Opacity / Not Normal |
| 1 | 00313ee0-9eaa-42f4-b0ab-c148ed3241cd | No Lung Opacity / Not Normal |
| 2 | 00322d4d-1c29-4943-afc9-b6754be640eb | No Lung Opacity / Not Normal |
| 3 | 003d8fa0-6bf1-40ed-b54c-ac657f8495c5 | Normal |
| 4 | 00436515-870c-4b36-a041-de91049b9ab4 | Lung Opacity |
Some information about the data field present in the 'stage_2_detailed_class_info.csv' are:
patientId - A patientId. Each patientId corresponds to a unique image class - Have three values depending what is the current state of the patient's lung: 'No Lung Opacity / Not Normal', 'Normal' and 'Lung Opacity'.
# Number of entries in class_label dataframe
print('The class_labels dataframe has {0} rows and {1} columns.'.format(class_labels.shape[0], class_labels.shape[1]))
The class_labels dataframe has 30227 rows and 2 columns.
# Number of duplicates in patients
print('Number of unique patientId are : {0}'.format(class_labels['patientId'].nunique()))
Number of unique patientId are : 26684
Thus, the dataset contains information about 26684 patients (which is same as that of the train_labels dataframe).
def get_feature_distribution(data, feature):
# Count for each label
label_counts = data[feature].value_counts()
# Count the number of items in each class
total_samples = len(data)
print("Feature: {}".format(feature))
for i in range(len(label_counts)):
label = label_counts.index[i]
count = label_counts.values[i]
percent = int((count / total_samples) * 10000) / 100
print("{0:<30s}: {1} which is {2}% of the total data in the dataset".format(label, count, percent))
get_feature_distribution(class_labels, 'class')
Feature: class No Lung Opacity / Not Normal : 11821 which is 39.1% of the total data in the dataset Lung Opacity : 9555 which is 31.61% of the total data in the dataset Normal : 8851 which is 29.28% of the total data in the dataset
plt.figure(figsize=(10,6))
class_labels['class'].value_counts().sort_index(ascending = False).plot(kind = 'pie', autopct = '%.0f%%', explode = [0.03,0.03,0.03]).set_ylabel('')
plt.show()
# Checking nulls in class_labels
print('Number of nulls in class columns : {0}'.format(class_labels['class'].isnull().sum()))
Number of nulls in class columns : 0
# Checking whether each patientId has only one type of class or not
class_labels.groupby(['patientId'])['class'].nunique().max()
1
# Merging the two dataset - 'train_labels' and 'class_labels'
training_data = pd.concat([train_labels, class_labels['class']], axis = 1)
print('After merging, the dataset looks like : ')
training_data.head()
After merging, the dataset looks like :
| patientId | x | y | width | height | Target | number_of_boxes | class | |
|---|---|---|---|---|---|---|---|---|
| 0 | 0004cfab-14fd-4e49-80ba-63a80b6bddd6 | NaN | NaN | NaN | NaN | 0 | 1 | No Lung Opacity / Not Normal |
| 1 | 00313ee0-9eaa-42f4-b0ab-c148ed3241cd | NaN | NaN | NaN | NaN | 0 | 1 | No Lung Opacity / Not Normal |
| 2 | 00322d4d-1c29-4943-afc9-b6754be640eb | NaN | NaN | NaN | NaN | 0 | 1 | No Lung Opacity / Not Normal |
| 3 | 003d8fa0-6bf1-40ed-b54c-ac657f8495c5 | NaN | NaN | NaN | NaN | 0 | 1 | Normal |
| 4 | 00436515-870c-4b36-a041-de91049b9ab4 | 264.0 | 152.0 | 213.0 | 379.0 | 1 | 2 | Lung Opacity |
print('After merge, the dataset has {0} rows and {1} columns.'.format(training_data.shape[0], training_data.shape[1]))
After merge, the dataset has 30227 rows and 8 columns.
Target and Class
The Classes and data are not balanced.
fig, ax = plt.subplots(nrows = 1, figsize = (12, 6))
temp = training_data.groupby('Target')['class'].value_counts()
data_target_class = pd.DataFrame(data = {'Values': temp.values}, index = temp.index).reset_index()
sns.barplot(ax = ax, x = 'Target', y = 'Values', hue = 'class', data = data_target_class, palette = 'Set3')
plt.title('Class and Target for Chest Exams')
plt.show()
Thus, Target = 1 is associated with only class = Lung Opacity whereas Target = 0 is associated with only class = No Lung Opacity / Not Normal as well as Normal.
Bound Box Distribution
fig, ax = plt.subplots(1, 1, figsize = (7, 7))
target_1 = training_data[training_data['Target'] == 1]
target_sample = target_1.sample(2000)
target_sample['xc'] = target_sample['x'] + target_sample['width'] / 2
target_sample['yc'] = target_sample['y'] + target_sample['height'] / 2
plt.title('Centers of Lung Opacity Rectangles (brown) over rectangles (yellow)\nSample Size: 2000')
target_sample.plot.scatter(x = 'xc', y = 'yc', xlim = (0, 1024), ylim = (0, 1024), ax = ax, alpha = 0.8, marker = '.', color = 'brown')
for i, crt_sample in target_sample.iterrows():
ax.add_patch(Rectangle(xy=(crt_sample['x'], crt_sample['y']),
width=crt_sample['width'],height=crt_sample['height'],alpha=3.5e-3, color="yellow"))
We can see that the centers for the bounding box are spread out evenly across the Lungs(red dots). Though a large portion of the bounding box have their centers at the centers of the Lung, but some centers of the box are also located at the edges of lung(yellow portion).
The images used in radiology are stored in DICOM format.
A DICOM file consists of a header and image data sets packed into a single file. The information within the header is organized as a constant and standardized series of tags. By extracting data from these tags one can access important information regarding the patient demographics, study parameters, etc. Below, we can see the details stored in the dicom files along with the images.
sample_patientId = train_labels['patientId'][0]
dcm_file = 'stage_2_train_images/'+'{}.dcm'.format(sample_patientId)
dcm_data = dicom.read_file(dcm_file)
print('Metadata of the image consists of : \n\n', dcm_data)
Metadata of the image consists of : Dataset.file_meta ------------------------------- (0002, 0000) File Meta Information Group Length UL: 202 (0002, 0001) File Meta Information Version OB: b'\x00\x01' (0002, 0002) Media Storage SOP Class UID UI: Secondary Capture Image Storage (0002, 0003) Media Storage SOP Instance UID UI: 1.2.276.0.7230010.3.1.4.8323329.28530.1517874485.775526 (0002, 0010) Transfer Syntax UID UI: JPEG Baseline (Process 1) (0002, 0012) Implementation Class UID UI: 1.2.276.0.7230010.3.0.3.6.0 (0002, 0013) Implementation Version Name SH: 'OFFIS_DCMTK_360' ------------------------------------------------- (0008, 0005) Specific Character Set CS: 'ISO_IR 100' (0008, 0016) SOP Class UID UI: Secondary Capture Image Storage (0008, 0018) SOP Instance UID UI: 1.2.276.0.7230010.3.1.4.8323329.28530.1517874485.775526 (0008, 0020) Study Date DA: '19010101' (0008, 0030) Study Time TM: '000000.00' (0008, 0050) Accession Number SH: '' (0008, 0060) Modality CS: 'CR' (0008, 0064) Conversion Type CS: 'WSD' (0008, 0090) Referring Physician's Name PN: '' (0008, 103e) Series Description LO: 'view: PA' (0010, 0010) Patient's Name PN: '0004cfab-14fd-4e49-80ba-63a80b6bddd6' (0010, 0020) Patient ID LO: '0004cfab-14fd-4e49-80ba-63a80b6bddd6' (0010, 0030) Patient's Birth Date DA: '' (0010, 0040) Patient's Sex CS: 'F' (0010, 1010) Patient's Age AS: '51' (0018, 0015) Body Part Examined CS: 'CHEST' (0018, 5101) View Position CS: 'PA' (0020, 000d) Study Instance UID UI: 1.2.276.0.7230010.3.1.2.8323329.28530.1517874485.775525 (0020, 000e) Series Instance UID UI: 1.2.276.0.7230010.3.1.3.8323329.28530.1517874485.775524 (0020, 0010) Study ID SH: '' (0020, 0011) Series Number IS: '1' (0020, 0013) Instance Number IS: '1' (0020, 0020) Patient Orientation CS: '' (0028, 0002) Samples per Pixel US: 1 (0028, 0004) Photometric Interpretation CS: 'MONOCHROME2' (0028, 0010) Rows US: 1024 (0028, 0011) Columns US: 1024 (0028, 0030) Pixel Spacing DS: [0.14300000000000002, 0.14300000000000002] (0028, 0100) Bits Allocated US: 8 (0028, 0101) Bits Stored US: 8 (0028, 0102) High Bit US: 7 (0028, 0103) Pixel Representation US: 0 (0028, 2110) Lossy Image Compression CS: '01' (0028, 2114) Lossy Image Compression Method CS: 'ISO_10918_1' (7fe0, 0010) Pixel Data OB: Array of 142006 elements
print('Number of images in training images folders are : {0}'.format(len(os.listdir('stage_2_train_images'))))
Number of images in training images folders are : 26684
Thus, we can see that in the training images folder we have just 26684 images which is same as that of unique patientId's present in either of the csv files. Thus, we can say that each of the unique patientId's present in either of the csv files corresponds to an image present in the folder.
training_image_path = 'stage_2_train_images/'
images = pd.DataFrame({'path': glob(os.path.join(training_image_path, '*.dcm'))})
images['patientId'] = images['path'].map(lambda x:os.path.splitext(os.path.basename(x))[0])
print('Columns in the training images dataframe : {0}'.format(list(images.columns)))
Columns in the training images dataframe : ['path', 'patientId']
# Merging the images dataframe with training_data dataframe
training_data = training_data.merge(images, on = 'patientId', how = 'left')
print('After merging the two dataframe, the training_data has {0} rows and {1} columns.'.format(training_data.shape[0], training_data.shape[1]))
After merging the two dataframe, the training_data has 30227 rows and 9 columns.
print('The training_data dataframe as of now stands like,')
training_data.head()
The training_data dataframe as of now stands like,
| patientId | x | y | width | height | Target | number_of_boxes | class | path | |
|---|---|---|---|---|---|---|---|---|---|
| 0 | 0004cfab-14fd-4e49-80ba-63a80b6bddd6 | NaN | NaN | NaN | NaN | 0 | 1 | No Lung Opacity / Not Normal | stage_2_train_images/0004cfab-14fd-4e49-80ba-6... |
| 1 | 00313ee0-9eaa-42f4-b0ab-c148ed3241cd | NaN | NaN | NaN | NaN | 0 | 1 | No Lung Opacity / Not Normal | stage_2_train_images/00313ee0-9eaa-42f4-b0ab-c... |
| 2 | 00322d4d-1c29-4943-afc9-b6754be640eb | NaN | NaN | NaN | NaN | 0 | 1 | No Lung Opacity / Not Normal | stage_2_train_images/00322d4d-1c29-4943-afc9-b... |
| 3 | 003d8fa0-6bf1-40ed-b54c-ac657f8495c5 | NaN | NaN | NaN | NaN | 0 | 1 | Normal | stage_2_train_images/003d8fa0-6bf1-40ed-b54c-a... |
| 4 | 00436515-870c-4b36-a041-de91049b9ab4 | 264.0 | 152.0 | 213.0 | 379.0 | 1 | 2 | Lung Opacity | stage_2_train_images/00436515-870c-4b36-a041-d... |
columns_to_add = ['Modality', 'PatientAge', 'PatientSex', 'BodyPartExamined', 'ViewPosition', 'ConversionType', 'Rows', 'Columns', 'PixelSpacing']
def parse_dicom_data(data_df, data_path):
for col in columns_to_add:
data_df[col] = None
image_names = os.listdir('stage_2_train_images/')
for i, img_name in tqdm_notebook(enumerate(image_names)):
imagepath = os.path.join('stage_2_train_images/', img_name)
data_img = dicom.read_file(imagepath)
idx = (data_df['patientId'] == data_img.PatientID)
data_df.loc[idx, 'Modality'] = data_img.Modality
data_df.loc[idx, 'PatientAge'] = pd.to_numeric(data_img.PatientAge)
data_df.loc[idx, 'PatientSex'] = data_img.PatientSex
data_df.loc[idx, 'BodyPartExamined'] = data_img.BodyPartExamined
data_df.loc[idx, 'ViewPosition'] = data_img.ViewPosition
data_df.loc[idx, 'ConversionType'] = data_img.ConversionType
data_df.loc[idx, 'Rows'] = data_img.Rows
data_df.loc[idx, 'Columns'] = data_img.Columns
data_df.loc[idx, 'PixelSpacing'] = str.format("{:4.3f}", data_img.PixelSpacing[0])
parse_dicom_data(training_data, 'stage_2_train_images/')
The data from the DICOM files are now read and put into the dataset.
Fileds like 'Modality', 'PatientAge', 'PatientSex', 'BodyPartExamined', etc have been added to the dataset for further processing.
print('So after parsing the information from the dicom images, our training_data dataframe has {0} rows and {1} columns and it looks like, \n'.format(training_data.shape[0], training_data.shape[1]))
training_data.head()
So after parsing the information from the dicom images, our training_data dataframe has 30227 rows and 18 columns and it looks like,
| patientId | x | y | width | height | Target | number_of_boxes | class | path | Modality | PatientAge | PatientSex | BodyPartExamined | ViewPosition | ConversionType | Rows | Columns | PixelSpacing | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 0004cfab-14fd-4e49-80ba-63a80b6bddd6 | NaN | NaN | NaN | NaN | 0 | 1 | No Lung Opacity / Not Normal | stage_2_train_images/0004cfab-14fd-4e49-80ba-6... | CR | 51 | F | CHEST | PA | WSD | 1024 | 1024 | 0.143 |
| 1 | 00313ee0-9eaa-42f4-b0ab-c148ed3241cd | NaN | NaN | NaN | NaN | 0 | 1 | No Lung Opacity / Not Normal | stage_2_train_images/00313ee0-9eaa-42f4-b0ab-c... | CR | 48 | F | CHEST | PA | WSD | 1024 | 1024 | 0.194 |
| 2 | 00322d4d-1c29-4943-afc9-b6754be640eb | NaN | NaN | NaN | NaN | 0 | 1 | No Lung Opacity / Not Normal | stage_2_train_images/00322d4d-1c29-4943-afc9-b... | CR | 19 | M | CHEST | AP | WSD | 1024 | 1024 | 0.168 |
| 3 | 003d8fa0-6bf1-40ed-b54c-ac657f8495c5 | NaN | NaN | NaN | NaN | 0 | 1 | Normal | stage_2_train_images/003d8fa0-6bf1-40ed-b54c-a... | CR | 28 | M | CHEST | PA | WSD | 1024 | 1024 | 0.143 |
| 4 | 00436515-870c-4b36-a041-de91049b9ab4 | 264.0 | 152.0 | 213.0 | 379.0 | 1 | 2 | Lung Opacity | stage_2_train_images/00436515-870c-4b36-a041-d... | CR | 32 | F | CHEST | AP | WSD | 1024 | 1024 | 0.139 |
# Saving the training_data for further use
training_data.to_pickle('training_data.pkl')
Going forward we will now use this pickle file as our training data.
In this section, we'll be analysing Final dataset to summarize their main characteristics, often using statistical graphics and other data visualization methods.
We have the final dataset as below, with 30227 rows and 18 columns.
training_data = pd.read_pickle('training_data.pkl')
print('Our training data consists of {0} rows and {1} columns and looks like as, \n'.format(training_data.shape[0], training_data.shape[1]))
training_data.head()
Our training data consists of 30227 rows and 18 columns and looks like as,
| patientId | x | y | width | height | Target | number_of_boxes | class | path | Modality | PatientAge | PatientSex | BodyPartExamined | ViewPosition | ConversionType | Rows | Columns | PixelSpacing | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 0004cfab-14fd-4e49-80ba-63a80b6bddd6 | NaN | NaN | NaN | NaN | 0 | 1 | No Lung Opacity / Not Normal | stage_2_train_images/0004cfab-14fd-4e49-80ba-6... | CR | 51 | F | CHEST | PA | WSD | 1024 | 1024 | 0.143 |
| 1 | 00313ee0-9eaa-42f4-b0ab-c148ed3241cd | NaN | NaN | NaN | NaN | 0 | 1 | No Lung Opacity / Not Normal | stage_2_train_images/00313ee0-9eaa-42f4-b0ab-c... | CR | 48 | F | CHEST | PA | WSD | 1024 | 1024 | 0.194 |
| 2 | 00322d4d-1c29-4943-afc9-b6754be640eb | NaN | NaN | NaN | NaN | 0 | 1 | No Lung Opacity / Not Normal | stage_2_train_images/00322d4d-1c29-4943-afc9-b... | CR | 19 | M | CHEST | AP | WSD | 1024 | 1024 | 0.168 |
| 3 | 003d8fa0-6bf1-40ed-b54c-ac657f8495c5 | NaN | NaN | NaN | NaN | 0 | 1 | Normal | stage_2_train_images/003d8fa0-6bf1-40ed-b54c-a... | CR | 28 | M | CHEST | PA | WSD | 1024 | 1024 | 0.143 |
| 4 | 00436515-870c-4b36-a041-de91049b9ab4 | 264.0 | 152.0 | 213.0 | 379.0 | 1 | 2 | Lung Opacity | stage_2_train_images/00436515-870c-4b36-a041-d... | CR | 32 | F | CHEST | AP | WSD | 1024 | 1024 | 0.139 |
Modality
print('Modality for the images obtained is : {0}'.format(training_data['Modality'].unique()[0]))
Modality for the images obtained is : CR
Body Part Examined
print('The images obtained are of {0} areas.'.format(training_data['BodyPartExamined'].unique()[0]))
The images obtained are of CHEST areas.
Understanding Different Positions:
get_feature_distribution(training_data, 'ViewPosition')
Feature: ViewPosition AP : 15297 which is 50.6% of the total data in the dataset PA : 14930 which is 49.39% of the total data in the dataset
ViewPosition
AP : 15297 which is 50.6% of the total data in the dataset
PA : 14930 which is 49.39% of the total data in the dataset
As seen above, two View Positions that are in the training dataset are AP (Anterior/Posterior) and PA (Posterior/Anterior).
These type of X-rays are mostly used to obtain the front-view. Apart from front-view, a lateral image is usually taken to complement the front-view.
print('The distribution of View Position when there is an evidence of Pneumonia :\n')
plt.figure(figsize=(10,6))
training_data.loc[training_data['Target'] == 1, 'ViewPosition'].value_counts().sort_index(ascending = False).plot(kind = 'pie', autopct = '%.0f%%', explode = [0.03,0.01], textprops = {"fontsize":13}).set_ylabel('')
plt.show()
The distribution of View Position when there is an evidence of Pneumonia :
print('Plot x and y centers of bounding box\n')
bboxes = training_data[training_data['Target'] == 1]
bboxes['xw'] = bboxes['x'] + bboxes['width']/2
bboxes['yh'] = bboxes['y'] + bboxes['height']/2
g = sns.jointplot(x = bboxes['xw'], y = bboxes['yh'], data = bboxes,
kind = 'hex', alpha = 0.5, size = 8)
plt.suptitle('Bounding Box location when there is an evidence of Pneumonia')
plt.tight_layout()
plt.subplots_adjust(top = 0.95)
Plot x and y centers of bounding box
def bboxes_scatter(data, color_point, color_window, text):
fig, ax = plt.subplots(1, 1, figsize = (7, 7))
plt.title('Plotting centers of Lung Opacity\n{}'.format(text))
data.plot.scatter(x = 'xw', y = 'yh', xlim = (0, 1024), ylim = (0, 1024), ax = ax, alpha = 0.8, marker = ".", color = color_point)
for i, crt_sample in data.iterrows():
ax.add_patch(Rectangle(xy = (crt_sample['x'], crt_sample['y']), width = crt_sample['width'], height = crt_sample['height'], alpha = 3.5e-3, color = color_window))
We now try to see the distribution of bounding boxes with different ViewPosition (AP and PA).
There are multiple outliers in both the cases.
data_PA = bboxes[bboxes['ViewPosition'] == 'PA'].sample(1000)
data_AP = bboxes[bboxes['ViewPosition'] == 'AP'].sample(1000)
bboxes_scatter(data_PA, 'green', 'yellow', 'ViewPosition = PA')
bboxes_scatter(data_AP, 'blue', 'red', 'ViewPosition = AP')
Conversion Type
print('Conversion Type for the data in Training Data : {0}'.format(training_data['ConversionType'].unique()[0]))
Conversion Type for the data in Training Data : WSD
Rows and Columns
print('The training images has {0} rows and {1} columns.'.format(training_data.Rows.unique()[0], training_data.Columns.unique()[0]))
The training images has 1024 rows and 1024 columns.
Patient Sex
def drawgraphs(data_file, columns, hue = False, width = 15, showdistribution = True):
if (hue):
print('Creating graph for : {} and {}\n'.format(columns, hue))
else:
print('Creating graph for : {}\n'.format(columns))
length = len(columns) * 6
total = float(len(data_file))
fig, axes = plt.subplots(nrows = len(columns) if len(columns) > 1 else 1, ncols = 1, figsize = (width, length))
for index, content in enumerate(columns):
plt.title(content)
currentaxes = 0
if (len(columns) > 1):
currentaxes = axes[index]
else:
currentaxes = axes
if (hue):
sns.countplot(x = columns[index], data = data_file, ax = currentaxes, hue = hue)
else:
sns.countplot(x = columns[index], data = data_file, ax = currentaxes)
if(showdistribution):
for p in (currentaxes.patches):
height = p.get_height()
if (height > 0 and total > 0):
currentaxes.text(p.get_x() + p.get_width()/2., height + 3, '{:1.2f}%'.format(100*height/total), ha = "center")
drawgraphs(data_file = training_data, columns = ['PatientSex'], hue = False, width = 10, showdistribution = True)
Creating graph for : ['PatientSex']
get_feature_distribution(training_data, 'PatientSex')
Feature: PatientSex M : 17216 which is 56.95% of the total data in the dataset F : 13011 which is 43.04% of the total data in the dataset
Thus, out of 30227 records, there are 17216 records of M (Male) and 13011 records of F (Female).
drawgraphs(data_file = training_data, columns = ['PatientSex'], hue = 'class', width = 10, showdistribution = True)
Creating graph for : ['PatientSex'] and class
drawgraphs(data_file = training_data, columns = ['PatientSex'], hue = 'Target', width = 10, showdistribution = True)
Creating graph for : ['PatientSex'] and Target
Thus, we can see that the number of Male patients suffering from Pneumonia is greater when compared with that of Females.
data_male = bboxes[bboxes['PatientSex'] == 'M'].sample(1000)
data_female = bboxes[bboxes['PatientSex'] == 'F'].sample(1000)
bboxes_scatter(data_male, "darkblue", "blue", "Patients Sex: Male")
bboxes_scatter(data_female, "red", "magenta", "Patients Sex: Female")
Patient Age
print('The minimum and maximum recorded age of the patients are {0} and {1} respectively.'.format(training_data['PatientAge'].min(), training_data['PatientAge'].max()))
The minimum and maximum recorded age of the patients are 1 and 155 respectively.
age_25 = np.percentile(training_data['PatientAge'], 25)
age_75 = np.percentile(training_data['PatientAge'], 75)
iqr_age = age_75 - age_25
cutoff_age = 1.5 * iqr_age
low_lim_age = age_25 - cutoff_age
upp_lim_age = age_75 + cutoff_age
outlier_age = [x for x in training_data['PatientAge'] if x < low_lim_age or x > upp_lim_age]
print('The number of outliers in `PatientAge` out of 30277 records are : {0}\n'.format(len(outlier_age)))
print('The ages which are in the outlier categories are : {0}\n'.format(outlier_age))
fig = plt.figure(figsize = (10, 6))
sns.boxplot(training_data['PatientAge'], orient = 'h').set_title('Outliers in PatientAge')
plt.show()
The number of outliers in `PatientAge` out of 30277 records are : 5 The ages which are in the outlier categories are : [148, 151, 153, 150, 155]
To confirm, we plot the distribution chart below to see the number of patients in a given age group.
We understand that maximum numbers of patients recorded belong to 56-58 years of age.
drawgraphs(data_file = training_data, columns = ['PatientAge'], width = 20, showdistribution = True)
Creating graph for : ['PatientAge']
print('Removing the outliers from `PatientAge`')
training_data['PatientAge'] = training_data['PatientAge'].clip(training_data['PatientAge'].min(), 100)
training_data['PatientAge'].describe().astype(int)
Removing the outliers from `PatientAge`
count 30227 unique 93 top 58 freq 955 Name: PatientAge, dtype: int64
print('Distribution of `PatientAge`: Overall and Target = 1\n')
fig = plt.figure(figsize = (10, 6))
ax = fig.add_subplot(121)
g = (sns.distplot(training_data['PatientAge']).set_title('Distribution of PatientAge'))
ax = fig.add_subplot(122)
g = (sns.distplot(training_data.loc[training_data['Target'] == 1, 'PatientAge']).set_title('Distribution of PatientAge vs Pnuemonia Evidence'))
Distribution of `PatientAge`: Overall and Target = 1
custom_array = np.linspace(0, 100, 11)
training_data['PatientAgeBins'] = pd.cut(training_data['PatientAge'], custom_array)
training_data['PatientAgeBins'].value_counts()
(50.0, 60.0] 7446 (40.0, 50.0] 5671 (60.0, 70.0] 4730 (30.0, 40.0] 4551 (20.0, 30.0] 3704 (10.0, 20.0] 1688 (70.0, 80.0] 1637 (0.0, 10.0] 515 (80.0, 90.0] 275 (90.0, 100.0] 10 Name: PatientAgeBins, dtype: int64
print('After adding the bin column, the dataset turns out to be :\n')
training_data.head()
After adding the bin column, the dataset turns out to be :
| patientId | x | y | width | height | Target | number_of_boxes | class | path | Modality | PatientAge | PatientSex | BodyPartExamined | ViewPosition | ConversionType | Rows | Columns | PixelSpacing | PatientAgeBins | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 0004cfab-14fd-4e49-80ba-63a80b6bddd6 | NaN | NaN | NaN | NaN | 0 | 1 | No Lung Opacity / Not Normal | stage_2_train_images/0004cfab-14fd-4e49-80ba-6... | CR | 51 | F | CHEST | PA | WSD | 1024 | 1024 | 0.143 | (50.0, 60.0] |
| 1 | 00313ee0-9eaa-42f4-b0ab-c148ed3241cd | NaN | NaN | NaN | NaN | 0 | 1 | No Lung Opacity / Not Normal | stage_2_train_images/00313ee0-9eaa-42f4-b0ab-c... | CR | 48 | F | CHEST | PA | WSD | 1024 | 1024 | 0.194 | (40.0, 50.0] |
| 2 | 00322d4d-1c29-4943-afc9-b6754be640eb | NaN | NaN | NaN | NaN | 0 | 1 | No Lung Opacity / Not Normal | stage_2_train_images/00322d4d-1c29-4943-afc9-b... | CR | 19 | M | CHEST | AP | WSD | 1024 | 1024 | 0.168 | (10.0, 20.0] |
| 3 | 003d8fa0-6bf1-40ed-b54c-ac657f8495c5 | NaN | NaN | NaN | NaN | 0 | 1 | Normal | stage_2_train_images/003d8fa0-6bf1-40ed-b54c-a... | CR | 28 | M | CHEST | PA | WSD | 1024 | 1024 | 0.143 | (20.0, 30.0] |
| 4 | 00436515-870c-4b36-a041-de91049b9ab4 | 264.0 | 152.0 | 213.0 | 379.0 | 1 | 2 | Lung Opacity | stage_2_train_images/00436515-870c-4b36-a041-d... | CR | 32 | F | CHEST | AP | WSD | 1024 | 1024 | 0.139 | (30.0, 40.0] |
drawgraphs(data_file = training_data, columns = ['PatientAgeBins'], width = 15, showdistribution = True)
Creating graph for : ['PatientAgeBins']
drawgraphs(data_file = training_data, columns = ['PatientAgeBins'], hue = 'PatientSex', width = 20, showdistribution = True)
Creating graph for : ['PatientAgeBins'] and PatientSex
drawgraphs(data_file = training_data, columns = ['PatientAgeBins'], hue = 'class', width = 20, showdistribution = True)
Creating graph for : ['PatientAgeBins'] and class
drawgraphs(data_file = training_data, columns = ['PatientAgeBins'], hue = 'Target', width = 20, showdistribution = True)
Creating graph for : ['PatientAgeBins'] and Target
data_age_19 = bboxes[bboxes['PatientAge'] < 20]
data_age_20_34 = bboxes[(bboxes['PatientAge'] >= 20) & (bboxes['PatientAge'] < 35)]
data_age_35_49 = bboxes[(bboxes['PatientAge'] >= 35) & (bboxes['PatientAge'] < 50)]
data_age_50_64 = bboxes[(bboxes['PatientAge'] >= 50) & (bboxes['PatientAge'] < 65)]
data_age_65 = bboxes[bboxes['PatientAge'] >= 65]
bboxes_scatter(data_age_19,'blue', 'red', 'Patient Age: 1-19 years')
bboxes_scatter(data_age_20_34, 'blue', 'red', 'Patient Age: 20-34 years')
bboxes_scatter(data_age_35_49, 'blue', 'red', 'Patient Age: 35-49 years')
bboxes_scatter(data_age_50_64, 'blue', 'red', 'Patient Age: 50-64 years')
bboxes_scatter(data_age_65, 'blue', 'red', 'Patient Age: 65+ years')
Plotting DICOM Images: To view the Training images, the DICOM image data is plotted.
def show_dicom_images(data, df, img_path):
img_data = list(data.T.to_dict().values())
f, ax = plt.subplots(3, 3, figsize = (16, 18))
for i, row in enumerate(img_data):
image = row['patientId'] + '.dcm'
path = os.path.join(img_path, image)
data = dicom.read_file(path)
rows = df[df['patientId'] == row['patientId']]
age = rows.PatientAge.unique().tolist()[0]
sex = data.PatientSex
part = data.BodyPartExamined
vp = data.ViewPosition
modality = data.Modality
data_img = dicom.dcmread(path)
ax[i//3, i%3].imshow(data_img.pixel_array, cmap = plt.cm.bone)
ax[i//3, i%3].axis('off')
ax[i//3, i%3].set_title('ID: {}\nAge: {}, Sex: {}, Part: {}, VP: {}, Modality: {}\nTarget: {}, Class: {}\nWindow: {}:{}:{}:{}'\
.format(row['patientId'], age, sex, part,
vp, modality, row['Target'],
row['class'], row['x'],
row['y'], row['width'],
row['height']))
box_data = list(rows.T.to_dict().values())
for j, row in enumerate(box_data):
ax[i//3, i%3].add_patch(Rectangle(xy = (row['x'], row['y']), width = row['width'], height = row['height'], color = 'red', alpha = 0.20))
plt.show()
# For Target=0
show_dicom_images(data = training_data.loc[(training_data['Target'] == 0)].sample(9), df = training_data, img_path = 'stage_2_train_images/')
As the above subplots are of the images which belong to either "Normal" or "No Lung Opacity / Not Normal", hence no bounding box is observed.
# For Target=1
show_dicom_images(data = training_data.loc[(training_data['Target'] == 1)].sample(9), df = training_data, img_path = 'stage_2_train_images/')
In the above subplots, we can see that the area covered by the box (in red colour) depicts the area of interest i.e., the area in which the opacity is observed in the Lungs.
The training dataset (both of the csv files and the training image folder) contains information of 26684 unique patients.
Out of these 26684 unique patients some of these have multiple entries in the both of the csv files.
Most of the recorded patient belong to Target = 0 (i.e., they don't have Pneumonia).
Some of the patients have more than one bounding box. The maximum being 4.
The classes "No Lung Opacity / Not Normal" and "Normal" is associated with Target = 0 whereas "Lung Opacity" belong to Target = 1
The images are present in dicom format, from which information like PatientAge, PatientSex, ViewPosition etc are obtained There are two ways from which images were obtained: AP and PA. The age ranges from 1-155 (which were further clipped to 100).
The centers of the bounding box are spread out over the entire region of the lungs. But there are some centers which are outliers.
New columns names are created and mapping done from original dataframe
Class-3 which has 'No Lung Opacity/Not Normal' is removed.
Balancing is done for classes as there are 3398 samples with lung opacity and 11465 with Normal. For instance, if we have 10 images with target label 1 and 100 images with target label 0 then below script will take all 10 images with label 1 and 10 randomly chosen images without replacement from the set of 100 images with label 0 and combine them to create a balanced set of 20 images. So this way we are able to get 1:1 distribution of classes.
We got images in HD resolution which is 1024x1024 so we need to downsize to 128x128,reason for using 128 size images is that we will be using Transfer Learning model called MobileNet pretrained on ImageNet Dataset. Since this problem is not only about prediction of classes (classification problem) also we need to predict coordinates of bounding boxes. So, we will be using UNET architecture for the problem solution. For this we created a user defined function and that function return training images array(X_train) with corresponding mask and target variable.
One Hot Encoding is performed on target variable.
For Classification, Sequential CNN, MobileNet, InceptionV3 are used and for UNet(Incombination with classifier) is used for detecting bounding boxes.
Model comparison is shown in the form of DataFrame and plotted the scores & losses.
In this section, we'll be building various models for Pneumonia detection.
folders = ['stage_2_train_images', 'stage_2_test_images']
path = '/content'
# Checking the number of files in folders
for f in folders:
print('No. of files in {0} : {1}'.format(f,len(os.listdir(os.path.join(path,f)))))
No. of files in stage_2_train_images : 26684 No. of files in stage_2_test_images : 3000
# Creating new column names and mapping from the original Dataframe
ColName = ['patientid','x1','y1','w1','h1','x2','y2','w2','h2','Target','Class','Gender']
# Iterating over patientId to get unique record
final_list=[]
for ids in np.unique(train_labels['patientId']):
meta_info = dicom.read_file('/content/stage_2_train_images/'+ids+'.dcm')
if len(list(train_labels['x'][train_labels['patientId']==ids]))>1: # We are considering only patientId with 2 bounding boxes
ls=[]
ls.append(ids)
ls.append(list(train_labels[train_labels['patientId']==ids]['x'])[0])
ls.append(list(train_labels[train_labels['patientId']==ids]['y'])[0])
ls.append(list(train_labels[train_labels['patientId']==ids]['width'])[0])
ls.append(list(train_labels[train_labels['patientId']==ids]['height'])[0])
ls.append(list(train_labels[train_labels['patientId']==ids]['x'])[1])
ls.append(list(train_labels[train_labels['patientId']==ids]['y'])[1])
ls.append(list(train_labels[train_labels['patientId']==ids]['width'])[1])
ls.append(list(train_labels[train_labels['patientId']==ids]['height'])[1])
ls.append(list(train_labels[train_labels['patientId']==ids]['Target'])[1])
ls.append(list(class_labels[class_labels['patientId']==ids]['class'])[0])
ls.append(meta_info.PatientSex)
final_list.append(ls)
else:
ls=[ids,0,0,0,0,0,0,0,0,0]
ls.append(list(class_labels[class_labels['patientId']==ids]['class'])[0])
ls.append(meta_info.PatientSex)
final_list.append(ls)
# Saving the dataframe
final_df = pd.DataFrame(final_list, columns=ColName)
# Removing the 3rd class (No Lung Opacity / Not Normal) from dataframe
final_df = final_df[final_df['Class'] != 'No Lung Opacity / Not Normal']
print('The final_df has {0} rows and {1} columns.\n'.format(final_df.shape[0],final_df.shape[1]))
# First 5 rows of dataframe
final_df.head(5)
The final_df has 14863 rows and 12 columns.
| patientid | x1 | y1 | w1 | h1 | x2 | y2 | w2 | h2 | Target | Class | Gender | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 000924cf-0f8d-42bd-9158-1af53881a557 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0 | Normal | F |
| 2 | 000db696-cf54-4385-b10b-6b16fbb3f985 | 316.0 | 318.0 | 170.0 | 478.0 | 660.0 | 375.0 | 146.0 | 402.0 | 1 | Lung Opacity | F |
| 3 | 000fe35a-2649-43d4-b027-e67796d412e0 | 570.0 | 282.0 | 269.0 | 409.0 | 83.0 | 227.0 | 296.0 | 438.0 | 1 | Lung Opacity | M |
| 4 | 001031d9-f904-4a23-b3e5-2c088acd19c6 | 66.0 | 160.0 | 373.0 | 608.0 | 552.0 | 164.0 | 376.0 | 676.0 | 1 | Lung Opacity | M |
| 5 | 0010f549-b242-4e94-87a8-57d79de215fc | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0 | Normal | M |
# Class distribution
final_df.groupby('Target').count()['Class']
Target 0 11465 1 3398 Name: Class, dtype: int64
Data pre-processing is first step for any AI ML problem and this step is very important for the prediction power of any model. Here we have images in dicom format so first we need to save them in an array so that we could train model with that and secondly we got images in HD resolution which is 1024x1024 so we need to downsize to 128x128,reason for using 128 size images is that we will be using Transfer Learning model called MobileNet pretrained on ImageNet Dataset. Since this problem is not only about prediction of classes (classification problem) also we need to predict coordinates of bounding boxes. So, we will be using UNET architecture for the problem solution. For this we created a user defined function and that function return training images array(X_train) with corresponding mask and target variable.
# Fuction which will return X_train, Mask and Target
def image_resizer_mask_generator(path, new_im_w, new_im_h):
listing=[x+'.dcm' for x in final_df['patientid'].unique()]
# Initialize X_train
X_train = np.zeros((len(listing), new_im_w, new_im_h, 3))
# Initialize Mask
mask = np.zeros((len(listing), new_im_w, new_im_h))
# Initialize Target
target = np.zeros(len(listing))
i=1
for index in range(len(listing)):
os.chdir(os.path.join(path,'stage_2_train_images'))
img = dicom.dcmread(listing[index]) # Read Dicom image
img_arr = img.pixel_array
img_arr = np.stack([img_arr]*3, axis=2) # Convert to 3 channels
orig_im_h, orig_im_w, channels = img_arr.shape
img_arr_resize = cv2.resize(img_arr, (new_im_w, new_im_h))
# Feed to X_train
X_train[index] = np.array(img_arr_resize, dtype=np.float32)
ids=listing[index].replace('.dcm','')
ids,x1,y1,w1,h1,x2,y2,w2,h2,tgt,label = [np.array(final_df[final_df['patientid']==ids])[0][i] for i in range(11)]
target[index] = tgt
if x1>0:
ratio_w=new_im_w/orig_im_w
ratio_h=new_im_h/orig_im_h
new_x1=int(x1*ratio_w)
new_y1=int(y1*ratio_h)
new_x2=int(x2*ratio_w)
new_y2=int(y2*ratio_h)
new_w1=int(w1*ratio_w)
new_h1=int(h1*ratio_h)
new_w2=int(w2*ratio_w)
new_h2=int(h2*ratio_h)
mask[index][new_y1:new_y1+new_h1, new_x1:new_x1+new_w1] = 1
mask[index][new_y2:new_y2+new_h2, new_x2:new_x2+new_w2] = 1
i+=1
return X_train, mask, target
(X_train, mask, target) = image_resizer_mask_generator('/content', 128, 128) # Resizing images to 128 x 128
Since we also observed that data distribution is imbalanced for this, we wrote a script which first separate out two classes images into separate variable then combine full set of images with target label 1 and a subset of randomly picked images with label 0(that subset must be of length equal to the length of the full set for target 1) For instance, if we have 10 images with target label 1 and 100 images with target label 0 then below script will take all 10 images with label 1 and 10 randomly chosen images without replacement from the set of 100 images with label 0 and combine them to create a balanced set of 20 images. So this way we are able to get 1:1 distribution of classes.
# Split target = 0
X_train_0 = X_train[target==0]
mask_0 = mask[target==0]
target_0 = target[target==0]
print('Length for Target=0 : {0}'.format(len(target_0)))
# Split target = 1
X_train_1 = X_train[target==1]
mask_1 = mask[target==1]
target_1 = target[target==1]
print('Length for Target=1 : {0}'.format(len(target_1)))
# Balance the Class using random function
X_train_balanced = np.zeros((2*len(target_1), X_train_1.shape[1], X_train_1.shape[2], X_train_1.shape[3]))
mask_balanced = np.zeros((2*len(target_1), mask_1.shape[1], mask_1.shape[2]))
target_balanced = np.zeros((2*len(target_1)))
index=0
for i in (np.random.choice(len(target_0), len(target_1), replace=False)):
X_train_balanced[index] = X_train_0[i]
mask_balanced[index] = mask_0[i]
target_balanced[index] = target_0[i]
index+=1
j=0
for idx in np.arange(len(target_1), X_train_balanced.shape[0], 1):
X_train_balanced[idx] = X_train_1[j]
mask_balanced[idx] = mask_1[j]
target_balanced[idx] = target_1[j]
j+=1
Length for Target=0 : 11465 Length for Target=1 : 3398
# Length after balancing
len(X_train_balanced), len(mask_balanced), len(target_balanced)
(6796, 6796, 6796)
# One hot encoding target variable
target_encoded = tf.keras.utils.to_categorical(target_balanced, num_classes=2)
# Shuffling the whole data
from sklearn.utils import shuffle
X_train_balanced, mask_balanced, target_encoded = shuffle(X_train_balanced, mask_balanced, target_encoded)
# Deleted variables to save memory
del X_train
del mask
del target
Before training the model, we need to divide whole set into training and evaluation set and here for Classification model we took first 5000 images as training set and remaining images is for evaluation of the performance on unseen data. And for the prediction of bounding boxes using UNET architecture we are using different set for training and evaluation. As we thought that if the person is normal then we don’t need to do any prediction of boxes for those images. Hence, we are feeding only those images which has some abnormality present and for them we need to highlight those affected area via bounding boxes. So, for UNET we created another set containing only images with target as 1 (Lung Opacity) and tried to predict mask and bounding boxes thereafter. For Evaluation set we took first 10 images and remaining images for training the UNET model.
# Taking first 5000 records for training and rest for evaluation
Train_X = X_train_balanced[0:5000,:,:,:]
Eval_X = X_train_balanced[5000:,:,:,:]
# Train_mask = mask_balanced[0:5000,:,:]
# Eval_mask = mask_balanced[5000:,:,:]
Train_y = target_encoded[0:5000]
Eval_y = target_encoded[5000:]
# For computation restrictions, we are training UNET only on images which has targer as 1 (Lung Opacity).
# Taking first 10 records for evaluation and rest for training
Train_X_unet = X_train_1[10:]
Eval_X_unet = X_train_1[0:10]
Train_mask_unet = mask_1[10:]
Eval_mask_unet = mask_1[0:10]
Highlights of Base Classifier Model:
Used TensorFlow 2.0 and Keras wrapper for building classification model.
Used Conv layers of 64 and 128 features with same padding.
Applied 2x2 kernel Maxpooling2D layers to make the model lighter.
Relu is used as activation function.
After flattening of feature vector, we applied 2 dense layers and last layer with SoftMax activation.
Achieved Evaluation accuracy obtained is 82%
# Function to plot accuracy and loss
def plot_accuracy_loss(history, model_type='UNET'):
plt.figure(figsize=(20,6))
# Plotting the Train accuracy, Validation accuracy vs number of Epochs
plt.subplot(1, 2, 1)
if(model_type=='CLF'):
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.ylabel('Accuracy')
else:
plt.plot(history.history['dice_coefficient'])
plt.plot(history.history['val_dice_coefficient'])
plt.ylabel('Dice Coefficient')
plt.xlabel('Epoch')
plt.title('Model Accuracy')
plt.legend(['Train Accuracy', 'Val Accuracy'], loc='lower right')
# Plotting the Train loss, Validation loss vs number of Epochs
plt.subplot(1, 2, 2)
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('Model Loss')
plt.ylabel('Loss')
plt.xlabel('Epoch')
plt.legend(['Train Loss', 'Val Loss'], loc='upper right')
plt.show()
# Model Evaluation
def evaluate_model(model, Eval_X, Eval_y, Train_X, Train_y, model_type='UNET'):
eval_results = model.evaluate(Eval_X, Eval_y)
train_results = model.evaluate(Train_X, Train_y)
if(model_type=='CLF'):
print('Accuracy on Eval set = {0:.2f}% & Loss on Eval set = {1:.4f}'.format(eval_results[1]*100, eval_results[0]))
train_acc = round(train_results[1]*100, 2)
eval_acc = round(eval_results[1]*100, 2)
else:
print('Dice Coefficient on Eval set = {0:.4f} & Loss on Eval set = {1:.4f}'.format(eval_results[1], eval_results[0]))
train_acc = round(train_results[1], 4)
eval_acc = round(eval_results[1], 4)
train_loss = round(train_results[0], 4)
eval_loss = round(eval_results[0], 4)
return eval_acc, eval_loss, train_acc, train_loss
# Dictionary to store the models, scores & loss
clf_model_scores = {'Model':[], 'Train Score':[], 'Eval Score':[], 'Train Loss':[], 'Eval Loss':[]}
unet_model_scores = {'Model':[], 'Train Score':[], 'Eval Score':[], 'Train Loss':[], 'Eval Loss':[]}
from sklearn.metrics import classification_report
from sklearn.metrics import confusion_matrix
def clf_report_and_conf_matrix(model, Eval_X, Eval_y):
prediction = model.predict(Eval_X) # Predict using the model
y_pred = np.argmax(prediction, axis=1)
y_actual = np.argmax(Eval_y, axis=1)
print('Classification Report:\n')
print(classification_report(y_actual, y_pred)) # Print Classification report
print('\n')
# Confusion matrix to evaluate the accuracy of a classification
cm = confusion_matrix(y_actual, y_pred)
df_cm = pd.DataFrame(cm,
index = [i for i in ['Non-Pneumonia','Pneumonia']],
columns = [i for i in ['Non-Pneumonia','Pneumonia']])
plt.figure(figsize = (7,5))
sns.heatmap(df_cm, annot=True ,fmt='g', cmap='coolwarm', linewidths=1) # Display confusion matrix
plt.xlabel('Predicted label')
plt.ylabel('True label')
plt.title('Confusion Matrix')
plt.show()
# Applying callbacks
early_stop = tf.keras.callbacks.EarlyStopping(monitor='loss', patience=5, mode='min')
reduce_lr = tf.keras.callbacks.ReduceLROnPlateau(monitor='loss', factor=0.2, patience=5, min_lr=1e-6, mode='min')
# checkpoint = tf.keras.callbacks.ModelCheckpoint('model-{loss:.2f}.h5', monitor='loss', save_best_only=True, save_weights_only=True, mode='min', save_freq=1)
# Define Dice Coefficient
def dice_coefficient(y_true, y_pred):
numerator = 2 * tf.reduce_sum(y_true * y_pred)
denominator = tf.reduce_sum(y_true + y_pred)
return numerator / (denominator + tf.keras.backend.epsilon())
# Define Loss function
def loss(y_true, y_pred):
return tf.keras.losses.binary_crossentropy(y_true, y_pred) - tf.keras.backend.log(dice_coefficient(y_true, y_pred) + tf.keras.backend.epsilon())
# Apply Pre-processing
Train_X_clf_1 = tf.keras.applications.mobilenet.preprocess_input(np.copy(Train_X))
Eval_X_clf_1 = tf.keras.applications.mobilenet.preprocess_input(np.copy(Eval_X))
# Basic CNN Model Architecture
# Sequential model
model_clf_1 = tf.keras.models.Sequential()
# Add the first convolution layer
model_clf_1.add(tf.keras.layers.Conv2D(32, kernel_size=(3, 3), strides=(1, 1), padding='same', activation='relu', input_shape=(128, 128, 3)))
# Add the first pooling layer
model_clf_1.add(tf.keras.layers.MaxPooling2D(pool_size=(2, 2)))
# Add the second convolution layer
model_clf_1.add(tf.keras.layers.Conv2D(64, (3, 3), padding='same', activation='relu'))
# Add the second pooling layer
model_clf_1.add(tf.keras.layers.MaxPooling2D(pool_size=(2, 2)))
# Add dropout layer
model_clf_1.add(tf.keras.layers.Dropout(0.2))
# Add the third convolution layer
model_clf_1.add(tf.keras.layers.Conv2D(128, (3, 3), padding='same', activation='relu'))
# Add the third pooling layer
model_clf_1.add(tf.keras.layers.MaxPooling2D(pool_size=(2, 2)))
# Add dropout layer
model_clf_1.add(tf.keras.layers.Dropout(0.2))
# Flatten the matrix into vector
model_clf_1.add(tf.keras.layers.Flatten())
model_clf_1.add(tf.keras.layers.Dropout(0.2))
# Add a dense (or fully connected) layer
model_clf_1.add(tf.keras.layers.Dense(32, activation='relu'))
# Add output layer
model_clf_1.add(tf.keras.layers.Dense(2, activation='softmax'))
# Model summary
model_clf_1.summary()
# Define optimizer and compile the model
model_clf_1.compile(
optimizer = tf.keras.optimizers.Adam(learning_rate=0.00001, beta_1=0.9, beta_2=0.999, epsilon=1e-07),
loss = 'categorical_crossentropy',
metrics = ['accuracy'])
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 128, 128, 32) 896
max_pooling2d (MaxPooling2D (None, 64, 64, 32) 0
)
conv2d_1 (Conv2D) (None, 64, 64, 64) 18496
max_pooling2d_1 (MaxPooling (None, 32, 32, 64) 0
2D)
dropout (Dropout) (None, 32, 32, 64) 0
conv2d_2 (Conv2D) (None, 32, 32, 128) 73856
max_pooling2d_2 (MaxPooling (None, 16, 16, 128) 0
2D)
dropout_1 (Dropout) (None, 16, 16, 128) 0
flatten (Flatten) (None, 32768) 0
dropout_2 (Dropout) (None, 32768) 0
dense (Dense) (None, 32) 1048608
dense_1 (Dense) (None, 2) 66
=================================================================
Total params: 1,141,922
Trainable params: 1,141,922
Non-trainable params: 0
_________________________________________________________________
# Plot the model
tf.keras.utils.plot_model(model_clf_1, show_shapes=True)
# Fit the model
history_clf_1 = model_clf_1.fit(Train_X_clf_1, Train_y, epochs=30, batch_size=32, validation_split=0.2, callbacks=[early_stop, reduce_lr])
Epoch 1/30 125/125 [==============================] - 17s 29ms/step - loss: 0.6268 - accuracy: 0.6700 - val_loss: 0.5488 - val_accuracy: 0.8160 - lr: 1.0000e-05 Epoch 2/30 125/125 [==============================] - 3s 21ms/step - loss: 0.5078 - accuracy: 0.7747 - val_loss: 0.4503 - val_accuracy: 0.8240 - lr: 1.0000e-05 Epoch 3/30 125/125 [==============================] - 3s 21ms/step - loss: 0.4741 - accuracy: 0.7908 - val_loss: 0.4306 - val_accuracy: 0.8310 - lr: 1.0000e-05 Epoch 4/30 125/125 [==============================] - 3s 21ms/step - loss: 0.4613 - accuracy: 0.7950 - val_loss: 0.4204 - val_accuracy: 0.8260 - lr: 1.0000e-05 Epoch 5/30 125/125 [==============================] - 3s 21ms/step - loss: 0.4502 - accuracy: 0.8008 - val_loss: 0.4174 - val_accuracy: 0.8280 - lr: 1.0000e-05 Epoch 6/30 125/125 [==============================] - 3s 21ms/step - loss: 0.4445 - accuracy: 0.8045 - val_loss: 0.4109 - val_accuracy: 0.8290 - lr: 1.0000e-05 Epoch 7/30 125/125 [==============================] - 3s 21ms/step - loss: 0.4402 - accuracy: 0.8040 - val_loss: 0.4101 - val_accuracy: 0.8320 - lr: 1.0000e-05 Epoch 8/30 125/125 [==============================] - 3s 22ms/step - loss: 0.4374 - accuracy: 0.8020 - val_loss: 0.4108 - val_accuracy: 0.8250 - lr: 1.0000e-05 Epoch 9/30 125/125 [==============================] - 3s 21ms/step - loss: 0.4314 - accuracy: 0.8145 - val_loss: 0.4143 - val_accuracy: 0.8200 - lr: 1.0000e-05 Epoch 10/30 125/125 [==============================] - 3s 22ms/step - loss: 0.4324 - accuracy: 0.8083 - val_loss: 0.4087 - val_accuracy: 0.8310 - lr: 1.0000e-05 Epoch 11/30 125/125 [==============================] - 3s 21ms/step - loss: 0.4274 - accuracy: 0.8133 - val_loss: 0.4085 - val_accuracy: 0.8280 - lr: 1.0000e-05 Epoch 12/30 125/125 [==============================] - 3s 22ms/step - loss: 0.4272 - accuracy: 0.8090 - val_loss: 0.4151 - val_accuracy: 0.8180 - lr: 1.0000e-05 Epoch 13/30 125/125 [==============================] - 3s 21ms/step - loss: 0.4244 - accuracy: 0.8102 - val_loss: 0.4119 - val_accuracy: 0.8210 - lr: 1.0000e-05 Epoch 14/30 125/125 [==============================] - 3s 21ms/step - loss: 0.4226 - accuracy: 0.8110 - val_loss: 0.4084 - val_accuracy: 0.8290 - lr: 1.0000e-05 Epoch 15/30 125/125 [==============================] - 3s 22ms/step - loss: 0.4222 - accuracy: 0.8105 - val_loss: 0.4066 - val_accuracy: 0.8260 - lr: 1.0000e-05 Epoch 16/30 125/125 [==============================] - 3s 22ms/step - loss: 0.4224 - accuracy: 0.8173 - val_loss: 0.4055 - val_accuracy: 0.8280 - lr: 1.0000e-05 Epoch 17/30 125/125 [==============================] - 3s 22ms/step - loss: 0.4205 - accuracy: 0.8145 - val_loss: 0.4051 - val_accuracy: 0.8290 - lr: 1.0000e-05 Epoch 18/30 125/125 [==============================] - 3s 22ms/step - loss: 0.4190 - accuracy: 0.8158 - val_loss: 0.4058 - val_accuracy: 0.8250 - lr: 1.0000e-05 Epoch 19/30 125/125 [==============================] - 3s 22ms/step - loss: 0.4179 - accuracy: 0.8100 - val_loss: 0.4040 - val_accuracy: 0.8260 - lr: 1.0000e-05 Epoch 20/30 125/125 [==============================] - 3s 21ms/step - loss: 0.4179 - accuracy: 0.8195 - val_loss: 0.4053 - val_accuracy: 0.8290 - lr: 1.0000e-05 Epoch 21/30 125/125 [==============================] - 3s 21ms/step - loss: 0.4125 - accuracy: 0.8188 - val_loss: 0.4055 - val_accuracy: 0.8330 - lr: 1.0000e-05 Epoch 22/30 125/125 [==============================] - 3s 21ms/step - loss: 0.4129 - accuracy: 0.8167 - val_loss: 0.4085 - val_accuracy: 0.8260 - lr: 1.0000e-05 Epoch 23/30 125/125 [==============================] - 3s 22ms/step - loss: 0.4110 - accuracy: 0.8210 - val_loss: 0.4047 - val_accuracy: 0.8260 - lr: 1.0000e-05 Epoch 24/30 125/125 [==============================] - 3s 22ms/step - loss: 0.4099 - accuracy: 0.8200 - val_loss: 0.4042 - val_accuracy: 0.8300 - lr: 1.0000e-05 Epoch 25/30 125/125 [==============================] - 3s 22ms/step - loss: 0.4095 - accuracy: 0.8177 - val_loss: 0.4065 - val_accuracy: 0.8240 - lr: 1.0000e-05 Epoch 26/30 125/125 [==============================] - 3s 22ms/step - loss: 0.4091 - accuracy: 0.8215 - val_loss: 0.3990 - val_accuracy: 0.8340 - lr: 1.0000e-05 Epoch 27/30 125/125 [==============================] - 3s 22ms/step - loss: 0.4053 - accuracy: 0.8232 - val_loss: 0.4029 - val_accuracy: 0.8290 - lr: 1.0000e-05 Epoch 28/30 125/125 [==============================] - 3s 22ms/step - loss: 0.4060 - accuracy: 0.8205 - val_loss: 0.4109 - val_accuracy: 0.8200 - lr: 1.0000e-05 Epoch 29/30 125/125 [==============================] - 3s 22ms/step - loss: 0.4042 - accuracy: 0.8217 - val_loss: 0.4013 - val_accuracy: 0.8280 - lr: 1.0000e-05 Epoch 30/30 125/125 [==============================] - 3s 22ms/step - loss: 0.4021 - accuracy: 0.8260 - val_loss: 0.4049 - val_accuracy: 0.8250 - lr: 1.0000e-05
# Plot accuracy and loss
plot_accuracy_loss(history=history_clf_1, model_type='CLF')
# Model Evaluation
eval_acc_clf_1, eval_loss_clf_1, train_acc_clf_1, train_loss_clf_1 = evaluate_model(model_clf_1, Eval_X_clf_1, Eval_y, Train_X_clf_1, Train_y, 'CLF')
57/57 [==============================] - 1s 11ms/step - loss: 0.4106 - accuracy: 0.8324 157/157 [==============================] - 2s 10ms/step - loss: 0.4027 - accuracy: 0.8272 Accuracy on Eval set = 83.24% & Loss on Eval set = 0.4106
# Classification report & Confusion matrix
clf_report_and_conf_matrix(model_clf_1, Eval_X_clf_1, Eval_y)
Classification Report:
precision recall f1-score support
0 0.82 0.85 0.84 895
1 0.85 0.81 0.83 901
accuracy 0.83 1796
macro avg 0.83 0.83 0.83 1796
weighted avg 0.83 0.83 0.83 1796
# Append the scores & loss
clf_model_scores['Model'].append('Basic CNN')
clf_model_scores['Train Score'].append(train_acc_clf_1)
clf_model_scores['Eval Score'].append(eval_acc_clf_1)
clf_model_scores['Train Loss'].append(train_loss_clf_1)
clf_model_scores['Eval Loss'].append(eval_loss_clf_1)
del Train_X_clf_1
del Eval_X_clf_1
Highlights of Transfer Learning Model:
We will use MobileNet pre-trained on ImageNet Dataset.
Then encode the features using MobileNet by removing the tail of dense layers.
UNet architecture is considered in the project as it is most preferable for MEDICAL IMAGES.
For computation restrictions, as mentioned above that for bounding boxes prediction, we are taking only images with target label 1 (Lung Opacity)
We will be feeding 128x128x3 dimension images into the model and predicting the mask of size 128x128 and similarly we are using the same MobileNet for classification prediction.
# Apply Pre-processing
Train_X_clf_2 = tf.keras.applications.mobilenet.preprocess_input(np.copy(Train_X))
Eval_X_clf_2 = tf.keras.applications.mobilenet.preprocess_input(np.copy(Eval_X))
# Instantiates the MobileNet architecture
model_clf_2_temp = tf.keras.applications.mobilenet.MobileNet(input_shape=(128, 128, 3), alpha=1.0, include_top=False, weights='imagenet')
# Previous layers are non trainable
for layer in model_clf_2_temp.layers:
layer.trainable=False
# Classification layers
y = tf.keras.layers.MaxPool2D(pool_size=(2, 2))(model_clf_2_temp.get_layer('conv_pw_13_relu').output)
y = tf.keras.layers.Flatten()(y)
y = tf.keras.layers.BatchNormalization()(y)
y = tf.keras.layers.Dense(512, activation='relu', kernel_regularizer=tf.keras.regularizers.l2(0.01), activity_regularizer=tf.keras.regularizers.l1(0.01))(y)
y = tf.keras.layers.Dropout(0.5)(y)
y = tf.keras.layers.Dense(64, activation='relu')(y)
y = tf.keras.layers.Dropout(0.5)(y)
y = tf.keras.layers.Dense(2, activation='softmax')(y)
model_clf_2 = tf.keras.Model(inputs=model_clf_2_temp.input, outputs=y)
# Model summary
model_clf_2.summary()
# Define optimizer and compile the model
model_clf_2.compile(
optimizer = tf.keras.optimizers.Adam(learning_rate=0.00001, beta_1=0.9, beta_2=0.999, epsilon=1e-07),
loss = 'categorical_crossentropy',
metrics = ['accuracy'])
Downloading data from https://storage.googleapis.com/tensorflow/keras-applications/mobilenet/mobilenet_1_0_128_tf_no_top.h5
17227776/17225924 [==============================] - 1s 0us/step
17235968/17225924 [==============================] - 1s 0us/step
Model: "model"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) [(None, 128, 128, 3)] 0
conv1 (Conv2D) (None, 64, 64, 32) 864
conv1_bn (BatchNormalizatio (None, 64, 64, 32) 128
n)
conv1_relu (ReLU) (None, 64, 64, 32) 0
conv_dw_1 (DepthwiseConv2D) (None, 64, 64, 32) 288
conv_dw_1_bn (BatchNormaliz (None, 64, 64, 32) 128
ation)
conv_dw_1_relu (ReLU) (None, 64, 64, 32) 0
conv_pw_1 (Conv2D) (None, 64, 64, 64) 2048
conv_pw_1_bn (BatchNormaliz (None, 64, 64, 64) 256
ation)
conv_pw_1_relu (ReLU) (None, 64, 64, 64) 0
conv_pad_2 (ZeroPadding2D) (None, 65, 65, 64) 0
conv_dw_2 (DepthwiseConv2D) (None, 32, 32, 64) 576
conv_dw_2_bn (BatchNormaliz (None, 32, 32, 64) 256
ation)
conv_dw_2_relu (ReLU) (None, 32, 32, 64) 0
conv_pw_2 (Conv2D) (None, 32, 32, 128) 8192
conv_pw_2_bn (BatchNormaliz (None, 32, 32, 128) 512
ation)
conv_pw_2_relu (ReLU) (None, 32, 32, 128) 0
conv_dw_3 (DepthwiseConv2D) (None, 32, 32, 128) 1152
conv_dw_3_bn (BatchNormaliz (None, 32, 32, 128) 512
ation)
conv_dw_3_relu (ReLU) (None, 32, 32, 128) 0
conv_pw_3 (Conv2D) (None, 32, 32, 128) 16384
conv_pw_3_bn (BatchNormaliz (None, 32, 32, 128) 512
ation)
conv_pw_3_relu (ReLU) (None, 32, 32, 128) 0
conv_pad_4 (ZeroPadding2D) (None, 33, 33, 128) 0
conv_dw_4 (DepthwiseConv2D) (None, 16, 16, 128) 1152
conv_dw_4_bn (BatchNormaliz (None, 16, 16, 128) 512
ation)
conv_dw_4_relu (ReLU) (None, 16, 16, 128) 0
conv_pw_4 (Conv2D) (None, 16, 16, 256) 32768
conv_pw_4_bn (BatchNormaliz (None, 16, 16, 256) 1024
ation)
conv_pw_4_relu (ReLU) (None, 16, 16, 256) 0
conv_dw_5 (DepthwiseConv2D) (None, 16, 16, 256) 2304
conv_dw_5_bn (BatchNormaliz (None, 16, 16, 256) 1024
ation)
conv_dw_5_relu (ReLU) (None, 16, 16, 256) 0
conv_pw_5 (Conv2D) (None, 16, 16, 256) 65536
conv_pw_5_bn (BatchNormaliz (None, 16, 16, 256) 1024
ation)
conv_pw_5_relu (ReLU) (None, 16, 16, 256) 0
conv_pad_6 (ZeroPadding2D) (None, 17, 17, 256) 0
conv_dw_6 (DepthwiseConv2D) (None, 8, 8, 256) 2304
conv_dw_6_bn (BatchNormaliz (None, 8, 8, 256) 1024
ation)
conv_dw_6_relu (ReLU) (None, 8, 8, 256) 0
conv_pw_6 (Conv2D) (None, 8, 8, 512) 131072
conv_pw_6_bn (BatchNormaliz (None, 8, 8, 512) 2048
ation)
conv_pw_6_relu (ReLU) (None, 8, 8, 512) 0
conv_dw_7 (DepthwiseConv2D) (None, 8, 8, 512) 4608
conv_dw_7_bn (BatchNormaliz (None, 8, 8, 512) 2048
ation)
conv_dw_7_relu (ReLU) (None, 8, 8, 512) 0
conv_pw_7 (Conv2D) (None, 8, 8, 512) 262144
conv_pw_7_bn (BatchNormaliz (None, 8, 8, 512) 2048
ation)
conv_pw_7_relu (ReLU) (None, 8, 8, 512) 0
conv_dw_8 (DepthwiseConv2D) (None, 8, 8, 512) 4608
conv_dw_8_bn (BatchNormaliz (None, 8, 8, 512) 2048
ation)
conv_dw_8_relu (ReLU) (None, 8, 8, 512) 0
conv_pw_8 (Conv2D) (None, 8, 8, 512) 262144
conv_pw_8_bn (BatchNormaliz (None, 8, 8, 512) 2048
ation)
conv_pw_8_relu (ReLU) (None, 8, 8, 512) 0
conv_dw_9 (DepthwiseConv2D) (None, 8, 8, 512) 4608
conv_dw_9_bn (BatchNormaliz (None, 8, 8, 512) 2048
ation)
conv_dw_9_relu (ReLU) (None, 8, 8, 512) 0
conv_pw_9 (Conv2D) (None, 8, 8, 512) 262144
conv_pw_9_bn (BatchNormaliz (None, 8, 8, 512) 2048
ation)
conv_pw_9_relu (ReLU) (None, 8, 8, 512) 0
conv_dw_10 (DepthwiseConv2D (None, 8, 8, 512) 4608
)
conv_dw_10_bn (BatchNormali (None, 8, 8, 512) 2048
zation)
conv_dw_10_relu (ReLU) (None, 8, 8, 512) 0
conv_pw_10 (Conv2D) (None, 8, 8, 512) 262144
conv_pw_10_bn (BatchNormali (None, 8, 8, 512) 2048
zation)
conv_pw_10_relu (ReLU) (None, 8, 8, 512) 0
conv_dw_11 (DepthwiseConv2D (None, 8, 8, 512) 4608
)
conv_dw_11_bn (BatchNormali (None, 8, 8, 512) 2048
zation)
conv_dw_11_relu (ReLU) (None, 8, 8, 512) 0
conv_pw_11 (Conv2D) (None, 8, 8, 512) 262144
conv_pw_11_bn (BatchNormali (None, 8, 8, 512) 2048
zation)
conv_pw_11_relu (ReLU) (None, 8, 8, 512) 0
conv_pad_12 (ZeroPadding2D) (None, 9, 9, 512) 0
conv_dw_12 (DepthwiseConv2D (None, 4, 4, 512) 4608
)
conv_dw_12_bn (BatchNormali (None, 4, 4, 512) 2048
zation)
conv_dw_12_relu (ReLU) (None, 4, 4, 512) 0
conv_pw_12 (Conv2D) (None, 4, 4, 1024) 524288
conv_pw_12_bn (BatchNormali (None, 4, 4, 1024) 4096
zation)
conv_pw_12_relu (ReLU) (None, 4, 4, 1024) 0
conv_dw_13 (DepthwiseConv2D (None, 4, 4, 1024) 9216
)
conv_dw_13_bn (BatchNormali (None, 4, 4, 1024) 4096
zation)
conv_dw_13_relu (ReLU) (None, 4, 4, 1024) 0
conv_pw_13 (Conv2D) (None, 4, 4, 1024) 1048576
conv_pw_13_bn (BatchNormali (None, 4, 4, 1024) 4096
zation)
conv_pw_13_relu (ReLU) (None, 4, 4, 1024) 0
max_pooling2d_3 (MaxPooling (None, 2, 2, 1024) 0
2D)
flatten_1 (Flatten) (None, 4096) 0
batch_normalization (BatchN (None, 4096) 16384
ormalization)
dense_2 (Dense) (None, 512) 2097664
dropout_3 (Dropout) (None, 512) 0
dense_3 (Dense) (None, 64) 32832
dropout_4 (Dropout) (None, 64) 0
dense_4 (Dense) (None, 2) 130
=================================================================
Total params: 5,375,874
Trainable params: 2,138,818
Non-trainable params: 3,237,056
_________________________________________________________________
# Plot the model
tf.keras.utils.plot_model(model_clf_2, show_shapes=True)
# Fit the model
history_clf_2 = model_clf_2.fit(Train_X_clf_2, Train_y, epochs=30, batch_size=32, validation_split=0.2, callbacks=[early_stop, reduce_lr])
Epoch 1/30 125/125 [==============================] - 5s 24ms/step - loss: 12.8759 - accuracy: 0.6102 - val_loss: 12.3733 - val_accuracy: 0.8020 - lr: 1.0000e-05 Epoch 2/30 125/125 [==============================] - 2s 17ms/step - loss: 12.4789 - accuracy: 0.7005 - val_loss: 12.0402 - val_accuracy: 0.8410 - lr: 1.0000e-05 Epoch 3/30 125/125 [==============================] - 2s 17ms/step - loss: 12.2509 - accuracy: 0.7427 - val_loss: 11.9661 - val_accuracy: 0.8460 - lr: 1.0000e-05 Epoch 4/30 125/125 [==============================] - 2s 17ms/step - loss: 12.1314 - accuracy: 0.7510 - val_loss: 11.9002 - val_accuracy: 0.8490 - lr: 1.0000e-05 Epoch 5/30 125/125 [==============================] - 2s 17ms/step - loss: 12.0101 - accuracy: 0.7667 - val_loss: 11.8164 - val_accuracy: 0.8540 - lr: 1.0000e-05 Epoch 6/30 125/125 [==============================] - 2s 17ms/step - loss: 11.8961 - accuracy: 0.7713 - val_loss: 11.7284 - val_accuracy: 0.8560 - lr: 1.0000e-05 Epoch 7/30 125/125 [==============================] - 2s 17ms/step - loss: 11.7845 - accuracy: 0.7772 - val_loss: 11.6363 - val_accuracy: 0.8590 - lr: 1.0000e-05 Epoch 8/30 125/125 [==============================] - 2s 17ms/step - loss: 11.6596 - accuracy: 0.7825 - val_loss: 11.5428 - val_accuracy: 0.8600 - lr: 1.0000e-05 Epoch 9/30 125/125 [==============================] - 2s 17ms/step - loss: 11.5832 - accuracy: 0.7820 - val_loss: 11.4503 - val_accuracy: 0.8600 - lr: 1.0000e-05 Epoch 10/30 125/125 [==============================] - 2s 17ms/step - loss: 11.4269 - accuracy: 0.7955 - val_loss: 11.3576 - val_accuracy: 0.8590 - lr: 1.0000e-05 Epoch 11/30 125/125 [==============================] - 2s 17ms/step - loss: 11.3334 - accuracy: 0.7958 - val_loss: 11.2628 - val_accuracy: 0.8590 - lr: 1.0000e-05 Epoch 12/30 125/125 [==============================] - 2s 17ms/step - loss: 11.2414 - accuracy: 0.7912 - val_loss: 11.1724 - val_accuracy: 0.8580 - lr: 1.0000e-05 Epoch 13/30 125/125 [==============================] - 2s 17ms/step - loss: 11.1146 - accuracy: 0.8043 - val_loss: 11.0866 - val_accuracy: 0.8590 - lr: 1.0000e-05 Epoch 14/30 125/125 [==============================] - 2s 17ms/step - loss: 11.0180 - accuracy: 0.8020 - val_loss: 10.9972 - val_accuracy: 0.8670 - lr: 1.0000e-05 Epoch 15/30 125/125 [==============================] - 2s 17ms/step - loss: 10.9200 - accuracy: 0.8058 - val_loss: 10.9036 - val_accuracy: 0.8670 - lr: 1.0000e-05 Epoch 16/30 125/125 [==============================] - 2s 17ms/step - loss: 10.8352 - accuracy: 0.8090 - val_loss: 10.8168 - val_accuracy: 0.8700 - lr: 1.0000e-05 Epoch 17/30 125/125 [==============================] - 2s 17ms/step - loss: 10.7136 - accuracy: 0.8142 - val_loss: 10.7242 - val_accuracy: 0.8710 - lr: 1.0000e-05 Epoch 18/30 125/125 [==============================] - 2s 17ms/step - loss: 10.6093 - accuracy: 0.8180 - val_loss: 10.6293 - val_accuracy: 0.8730 - lr: 1.0000e-05 Epoch 19/30 125/125 [==============================] - 2s 17ms/step - loss: 10.5384 - accuracy: 0.8177 - val_loss: 10.5407 - val_accuracy: 0.8690 - lr: 1.0000e-05 Epoch 20/30 125/125 [==============================] - 2s 17ms/step - loss: 10.4280 - accuracy: 0.8230 - val_loss: 10.4526 - val_accuracy: 0.8690 - lr: 1.0000e-05 Epoch 21/30 125/125 [==============================] - 2s 17ms/step - loss: 10.3264 - accuracy: 0.8278 - val_loss: 10.3611 - val_accuracy: 0.8680 - lr: 1.0000e-05 Epoch 22/30 125/125 [==============================] - 2s 17ms/step - loss: 10.2416 - accuracy: 0.8167 - val_loss: 10.2699 - val_accuracy: 0.8680 - lr: 1.0000e-05 Epoch 23/30 125/125 [==============================] - 2s 17ms/step - loss: 10.1273 - accuracy: 0.8288 - val_loss: 10.1833 - val_accuracy: 0.8700 - lr: 1.0000e-05 Epoch 24/30 125/125 [==============================] - 2s 17ms/step - loss: 10.0380 - accuracy: 0.8267 - val_loss: 10.0962 - val_accuracy: 0.8690 - lr: 1.0000e-05 Epoch 25/30 125/125 [==============================] - 2s 17ms/step - loss: 9.9269 - accuracy: 0.8388 - val_loss: 10.0070 - val_accuracy: 0.8710 - lr: 1.0000e-05 Epoch 26/30 125/125 [==============================] - 2s 17ms/step - loss: 9.8350 - accuracy: 0.8403 - val_loss: 9.9135 - val_accuracy: 0.8690 - lr: 1.0000e-05 Epoch 27/30 125/125 [==============================] - 2s 17ms/step - loss: 9.7255 - accuracy: 0.8440 - val_loss: 9.8236 - val_accuracy: 0.8700 - lr: 1.0000e-05 Epoch 28/30 125/125 [==============================] - 2s 17ms/step - loss: 9.6387 - accuracy: 0.8485 - val_loss: 9.7276 - val_accuracy: 0.8730 - lr: 1.0000e-05 Epoch 29/30 125/125 [==============================] - 2s 17ms/step - loss: 9.5492 - accuracy: 0.8445 - val_loss: 9.6365 - val_accuracy: 0.8690 - lr: 1.0000e-05 Epoch 30/30 125/125 [==============================] - 2s 17ms/step - loss: 9.4489 - accuracy: 0.8522 - val_loss: 9.5431 - val_accuracy: 0.8690 - lr: 1.0000e-05
# Plot accuracy and loss
plot_accuracy_loss(history=history_clf_2, model_type='CLF')
# Model Evaluation
eval_acc_clf_2, eval_loss_clf_2, train_acc_clf_2, train_loss_clf_2 = evaluate_model(model_clf_2, Eval_X_clf_2, Eval_y, Train_X_clf_2, Train_y, 'CLF')
57/57 [==============================] - 1s 16ms/step - loss: 9.5320 - accuracy: 0.8686 157/157 [==============================] - 2s 14ms/step - loss: 9.3639 - accuracy: 0.9284 Accuracy on Eval set = 86.86% & Loss on Eval set = 9.5320
# Classification report & Confusion matrix
clf_report_and_conf_matrix(model_clf_2, Eval_X_clf_2, Eval_y)
Classification Report:
precision recall f1-score support
0 0.89 0.84 0.86 895
1 0.85 0.90 0.87 901
accuracy 0.87 1796
macro avg 0.87 0.87 0.87 1796
weighted avg 0.87 0.87 0.87 1796
# Append the scores & loss
clf_model_scores['Model'].append('MobileNet')
clf_model_scores['Train Score'].append(train_acc_clf_2)
clf_model_scores['Eval Score'].append(eval_acc_clf_2)
clf_model_scores['Train Loss'].append(train_loss_clf_2)
clf_model_scores['Eval Loss'].append(eval_loss_clf_2)
del Train_X_clf_2
del Eval_X_clf_2
# Apply Pre-processing
Train_X_clf_3 = tf.keras.applications.inception_v3.preprocess_input(np.copy(Train_X))
Eval_X_clf_3 = tf.keras.applications.inception_v3.preprocess_input(np.copy(Eval_X))
# Instantiates the InceptionV3 architecture
model_clf_3_temp = tf.keras.applications.inception_v3.InceptionV3(input_shape=(128, 128, 3), include_top=False, weights='imagenet')
# Previous layers are non trainable
for layer in model_clf_3_temp.layers:
layer.trainable=False
input = model_clf_3_temp.input
x = tf.keras.layers.Flatten()(model_clf_3_temp.output)
x = tf.keras.layers.Dense(1024, activation='relu')(x)
x = tf.keras.layers.Dropout(0.2)(x)
output = tf.keras.layers.Dense(2, activation='softmax')(x)
model_clf_3 = tf.keras.Model(inputs=input, outputs=output)
# Model summary
model_clf_3.summary()
# Define optimizer and compile the model
model_clf_3.compile(
optimizer = tf.keras.optimizers.Adam(learning_rate=0.0001, beta_1=0.9, beta_2=0.999, epsilon=1e-07),
loss = 'categorical_crossentropy',
metrics = ['accuracy'])
Downloading data from https://storage.googleapis.com/tensorflow/keras-applications/inception_v3/inception_v3_weights_tf_dim_ordering_tf_kernels_notop.h5
87916544/87910968 [==============================] - 2s 0us/step
87924736/87910968 [==============================] - 2s 0us/step
Model: "model_1"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_2 (InputLayer) [(None, 128, 128, 3 0 []
)]
conv2d_3 (Conv2D) (None, 63, 63, 32) 864 ['input_2[0][0]']
batch_normalization_1 (BatchNo (None, 63, 63, 32) 96 ['conv2d_3[0][0]']
rmalization)
activation (Activation) (None, 63, 63, 32) 0 ['batch_normalization_1[0][0]']
conv2d_4 (Conv2D) (None, 61, 61, 32) 9216 ['activation[0][0]']
batch_normalization_2 (BatchNo (None, 61, 61, 32) 96 ['conv2d_4[0][0]']
rmalization)
activation_1 (Activation) (None, 61, 61, 32) 0 ['batch_normalization_2[0][0]']
conv2d_5 (Conv2D) (None, 61, 61, 64) 18432 ['activation_1[0][0]']
batch_normalization_3 (BatchNo (None, 61, 61, 64) 192 ['conv2d_5[0][0]']
rmalization)
activation_2 (Activation) (None, 61, 61, 64) 0 ['batch_normalization_3[0][0]']
max_pooling2d_4 (MaxPooling2D) (None, 30, 30, 64) 0 ['activation_2[0][0]']
conv2d_6 (Conv2D) (None, 30, 30, 80) 5120 ['max_pooling2d_4[0][0]']
batch_normalization_4 (BatchNo (None, 30, 30, 80) 240 ['conv2d_6[0][0]']
rmalization)
activation_3 (Activation) (None, 30, 30, 80) 0 ['batch_normalization_4[0][0]']
conv2d_7 (Conv2D) (None, 28, 28, 192) 138240 ['activation_3[0][0]']
batch_normalization_5 (BatchNo (None, 28, 28, 192) 576 ['conv2d_7[0][0]']
rmalization)
activation_4 (Activation) (None, 28, 28, 192) 0 ['batch_normalization_5[0][0]']
max_pooling2d_5 (MaxPooling2D) (None, 13, 13, 192) 0 ['activation_4[0][0]']
conv2d_11 (Conv2D) (None, 13, 13, 64) 12288 ['max_pooling2d_5[0][0]']
batch_normalization_9 (BatchNo (None, 13, 13, 64) 192 ['conv2d_11[0][0]']
rmalization)
activation_8 (Activation) (None, 13, 13, 64) 0 ['batch_normalization_9[0][0]']
conv2d_9 (Conv2D) (None, 13, 13, 48) 9216 ['max_pooling2d_5[0][0]']
conv2d_12 (Conv2D) (None, 13, 13, 96) 55296 ['activation_8[0][0]']
batch_normalization_7 (BatchNo (None, 13, 13, 48) 144 ['conv2d_9[0][0]']
rmalization)
batch_normalization_10 (BatchN (None, 13, 13, 96) 288 ['conv2d_12[0][0]']
ormalization)
activation_6 (Activation) (None, 13, 13, 48) 0 ['batch_normalization_7[0][0]']
activation_9 (Activation) (None, 13, 13, 96) 0 ['batch_normalization_10[0][0]']
average_pooling2d (AveragePool (None, 13, 13, 192) 0 ['max_pooling2d_5[0][0]']
ing2D)
conv2d_8 (Conv2D) (None, 13, 13, 64) 12288 ['max_pooling2d_5[0][0]']
conv2d_10 (Conv2D) (None, 13, 13, 64) 76800 ['activation_6[0][0]']
conv2d_13 (Conv2D) (None, 13, 13, 96) 82944 ['activation_9[0][0]']
conv2d_14 (Conv2D) (None, 13, 13, 32) 6144 ['average_pooling2d[0][0]']
batch_normalization_6 (BatchNo (None, 13, 13, 64) 192 ['conv2d_8[0][0]']
rmalization)
batch_normalization_8 (BatchNo (None, 13, 13, 64) 192 ['conv2d_10[0][0]']
rmalization)
batch_normalization_11 (BatchN (None, 13, 13, 96) 288 ['conv2d_13[0][0]']
ormalization)
batch_normalization_12 (BatchN (None, 13, 13, 32) 96 ['conv2d_14[0][0]']
ormalization)
activation_5 (Activation) (None, 13, 13, 64) 0 ['batch_normalization_6[0][0]']
activation_7 (Activation) (None, 13, 13, 64) 0 ['batch_normalization_8[0][0]']
activation_10 (Activation) (None, 13, 13, 96) 0 ['batch_normalization_11[0][0]']
activation_11 (Activation) (None, 13, 13, 32) 0 ['batch_normalization_12[0][0]']
mixed0 (Concatenate) (None, 13, 13, 256) 0 ['activation_5[0][0]',
'activation_7[0][0]',
'activation_10[0][0]',
'activation_11[0][0]']
conv2d_18 (Conv2D) (None, 13, 13, 64) 16384 ['mixed0[0][0]']
batch_normalization_16 (BatchN (None, 13, 13, 64) 192 ['conv2d_18[0][0]']
ormalization)
activation_15 (Activation) (None, 13, 13, 64) 0 ['batch_normalization_16[0][0]']
conv2d_16 (Conv2D) (None, 13, 13, 48) 12288 ['mixed0[0][0]']
conv2d_19 (Conv2D) (None, 13, 13, 96) 55296 ['activation_15[0][0]']
batch_normalization_14 (BatchN (None, 13, 13, 48) 144 ['conv2d_16[0][0]']
ormalization)
batch_normalization_17 (BatchN (None, 13, 13, 96) 288 ['conv2d_19[0][0]']
ormalization)
activation_13 (Activation) (None, 13, 13, 48) 0 ['batch_normalization_14[0][0]']
activation_16 (Activation) (None, 13, 13, 96) 0 ['batch_normalization_17[0][0]']
average_pooling2d_1 (AveragePo (None, 13, 13, 256) 0 ['mixed0[0][0]']
oling2D)
conv2d_15 (Conv2D) (None, 13, 13, 64) 16384 ['mixed0[0][0]']
conv2d_17 (Conv2D) (None, 13, 13, 64) 76800 ['activation_13[0][0]']
conv2d_20 (Conv2D) (None, 13, 13, 96) 82944 ['activation_16[0][0]']
conv2d_21 (Conv2D) (None, 13, 13, 64) 16384 ['average_pooling2d_1[0][0]']
batch_normalization_13 (BatchN (None, 13, 13, 64) 192 ['conv2d_15[0][0]']
ormalization)
batch_normalization_15 (BatchN (None, 13, 13, 64) 192 ['conv2d_17[0][0]']
ormalization)
batch_normalization_18 (BatchN (None, 13, 13, 96) 288 ['conv2d_20[0][0]']
ormalization)
batch_normalization_19 (BatchN (None, 13, 13, 64) 192 ['conv2d_21[0][0]']
ormalization)
activation_12 (Activation) (None, 13, 13, 64) 0 ['batch_normalization_13[0][0]']
activation_14 (Activation) (None, 13, 13, 64) 0 ['batch_normalization_15[0][0]']
activation_17 (Activation) (None, 13, 13, 96) 0 ['batch_normalization_18[0][0]']
activation_18 (Activation) (None, 13, 13, 64) 0 ['batch_normalization_19[0][0]']
mixed1 (Concatenate) (None, 13, 13, 288) 0 ['activation_12[0][0]',
'activation_14[0][0]',
'activation_17[0][0]',
'activation_18[0][0]']
conv2d_25 (Conv2D) (None, 13, 13, 64) 18432 ['mixed1[0][0]']
batch_normalization_23 (BatchN (None, 13, 13, 64) 192 ['conv2d_25[0][0]']
ormalization)
activation_22 (Activation) (None, 13, 13, 64) 0 ['batch_normalization_23[0][0]']
conv2d_23 (Conv2D) (None, 13, 13, 48) 13824 ['mixed1[0][0]']
conv2d_26 (Conv2D) (None, 13, 13, 96) 55296 ['activation_22[0][0]']
batch_normalization_21 (BatchN (None, 13, 13, 48) 144 ['conv2d_23[0][0]']
ormalization)
batch_normalization_24 (BatchN (None, 13, 13, 96) 288 ['conv2d_26[0][0]']
ormalization)
activation_20 (Activation) (None, 13, 13, 48) 0 ['batch_normalization_21[0][0]']
activation_23 (Activation) (None, 13, 13, 96) 0 ['batch_normalization_24[0][0]']
average_pooling2d_2 (AveragePo (None, 13, 13, 288) 0 ['mixed1[0][0]']
oling2D)
conv2d_22 (Conv2D) (None, 13, 13, 64) 18432 ['mixed1[0][0]']
conv2d_24 (Conv2D) (None, 13, 13, 64) 76800 ['activation_20[0][0]']
conv2d_27 (Conv2D) (None, 13, 13, 96) 82944 ['activation_23[0][0]']
conv2d_28 (Conv2D) (None, 13, 13, 64) 18432 ['average_pooling2d_2[0][0]']
batch_normalization_20 (BatchN (None, 13, 13, 64) 192 ['conv2d_22[0][0]']
ormalization)
batch_normalization_22 (BatchN (None, 13, 13, 64) 192 ['conv2d_24[0][0]']
ormalization)
batch_normalization_25 (BatchN (None, 13, 13, 96) 288 ['conv2d_27[0][0]']
ormalization)
batch_normalization_26 (BatchN (None, 13, 13, 64) 192 ['conv2d_28[0][0]']
ormalization)
activation_19 (Activation) (None, 13, 13, 64) 0 ['batch_normalization_20[0][0]']
activation_21 (Activation) (None, 13, 13, 64) 0 ['batch_normalization_22[0][0]']
activation_24 (Activation) (None, 13, 13, 96) 0 ['batch_normalization_25[0][0]']
activation_25 (Activation) (None, 13, 13, 64) 0 ['batch_normalization_26[0][0]']
mixed2 (Concatenate) (None, 13, 13, 288) 0 ['activation_19[0][0]',
'activation_21[0][0]',
'activation_24[0][0]',
'activation_25[0][0]']
conv2d_30 (Conv2D) (None, 13, 13, 64) 18432 ['mixed2[0][0]']
batch_normalization_28 (BatchN (None, 13, 13, 64) 192 ['conv2d_30[0][0]']
ormalization)
activation_27 (Activation) (None, 13, 13, 64) 0 ['batch_normalization_28[0][0]']
conv2d_31 (Conv2D) (None, 13, 13, 96) 55296 ['activation_27[0][0]']
batch_normalization_29 (BatchN (None, 13, 13, 96) 288 ['conv2d_31[0][0]']
ormalization)
activation_28 (Activation) (None, 13, 13, 96) 0 ['batch_normalization_29[0][0]']
conv2d_29 (Conv2D) (None, 6, 6, 384) 995328 ['mixed2[0][0]']
conv2d_32 (Conv2D) (None, 6, 6, 96) 82944 ['activation_28[0][0]']
batch_normalization_27 (BatchN (None, 6, 6, 384) 1152 ['conv2d_29[0][0]']
ormalization)
batch_normalization_30 (BatchN (None, 6, 6, 96) 288 ['conv2d_32[0][0]']
ormalization)
activation_26 (Activation) (None, 6, 6, 384) 0 ['batch_normalization_27[0][0]']
activation_29 (Activation) (None, 6, 6, 96) 0 ['batch_normalization_30[0][0]']
max_pooling2d_6 (MaxPooling2D) (None, 6, 6, 288) 0 ['mixed2[0][0]']
mixed3 (Concatenate) (None, 6, 6, 768) 0 ['activation_26[0][0]',
'activation_29[0][0]',
'max_pooling2d_6[0][0]']
conv2d_37 (Conv2D) (None, 6, 6, 128) 98304 ['mixed3[0][0]']
batch_normalization_35 (BatchN (None, 6, 6, 128) 384 ['conv2d_37[0][0]']
ormalization)
activation_34 (Activation) (None, 6, 6, 128) 0 ['batch_normalization_35[0][0]']
conv2d_38 (Conv2D) (None, 6, 6, 128) 114688 ['activation_34[0][0]']
batch_normalization_36 (BatchN (None, 6, 6, 128) 384 ['conv2d_38[0][0]']
ormalization)
activation_35 (Activation) (None, 6, 6, 128) 0 ['batch_normalization_36[0][0]']
conv2d_34 (Conv2D) (None, 6, 6, 128) 98304 ['mixed3[0][0]']
conv2d_39 (Conv2D) (None, 6, 6, 128) 114688 ['activation_35[0][0]']
batch_normalization_32 (BatchN (None, 6, 6, 128) 384 ['conv2d_34[0][0]']
ormalization)
batch_normalization_37 (BatchN (None, 6, 6, 128) 384 ['conv2d_39[0][0]']
ormalization)
activation_31 (Activation) (None, 6, 6, 128) 0 ['batch_normalization_32[0][0]']
activation_36 (Activation) (None, 6, 6, 128) 0 ['batch_normalization_37[0][0]']
conv2d_35 (Conv2D) (None, 6, 6, 128) 114688 ['activation_31[0][0]']
conv2d_40 (Conv2D) (None, 6, 6, 128) 114688 ['activation_36[0][0]']
batch_normalization_33 (BatchN (None, 6, 6, 128) 384 ['conv2d_35[0][0]']
ormalization)
batch_normalization_38 (BatchN (None, 6, 6, 128) 384 ['conv2d_40[0][0]']
ormalization)
activation_32 (Activation) (None, 6, 6, 128) 0 ['batch_normalization_33[0][0]']
activation_37 (Activation) (None, 6, 6, 128) 0 ['batch_normalization_38[0][0]']
average_pooling2d_3 (AveragePo (None, 6, 6, 768) 0 ['mixed3[0][0]']
oling2D)
conv2d_33 (Conv2D) (None, 6, 6, 192) 147456 ['mixed3[0][0]']
conv2d_36 (Conv2D) (None, 6, 6, 192) 172032 ['activation_32[0][0]']
conv2d_41 (Conv2D) (None, 6, 6, 192) 172032 ['activation_37[0][0]']
conv2d_42 (Conv2D) (None, 6, 6, 192) 147456 ['average_pooling2d_3[0][0]']
batch_normalization_31 (BatchN (None, 6, 6, 192) 576 ['conv2d_33[0][0]']
ormalization)
batch_normalization_34 (BatchN (None, 6, 6, 192) 576 ['conv2d_36[0][0]']
ormalization)
batch_normalization_39 (BatchN (None, 6, 6, 192) 576 ['conv2d_41[0][0]']
ormalization)
batch_normalization_40 (BatchN (None, 6, 6, 192) 576 ['conv2d_42[0][0]']
ormalization)
activation_30 (Activation) (None, 6, 6, 192) 0 ['batch_normalization_31[0][0]']
activation_33 (Activation) (None, 6, 6, 192) 0 ['batch_normalization_34[0][0]']
activation_38 (Activation) (None, 6, 6, 192) 0 ['batch_normalization_39[0][0]']
activation_39 (Activation) (None, 6, 6, 192) 0 ['batch_normalization_40[0][0]']
mixed4 (Concatenate) (None, 6, 6, 768) 0 ['activation_30[0][0]',
'activation_33[0][0]',
'activation_38[0][0]',
'activation_39[0][0]']
conv2d_47 (Conv2D) (None, 6, 6, 160) 122880 ['mixed4[0][0]']
batch_normalization_45 (BatchN (None, 6, 6, 160) 480 ['conv2d_47[0][0]']
ormalization)
activation_44 (Activation) (None, 6, 6, 160) 0 ['batch_normalization_45[0][0]']
conv2d_48 (Conv2D) (None, 6, 6, 160) 179200 ['activation_44[0][0]']
batch_normalization_46 (BatchN (None, 6, 6, 160) 480 ['conv2d_48[0][0]']
ormalization)
activation_45 (Activation) (None, 6, 6, 160) 0 ['batch_normalization_46[0][0]']
conv2d_44 (Conv2D) (None, 6, 6, 160) 122880 ['mixed4[0][0]']
conv2d_49 (Conv2D) (None, 6, 6, 160) 179200 ['activation_45[0][0]']
batch_normalization_42 (BatchN (None, 6, 6, 160) 480 ['conv2d_44[0][0]']
ormalization)
batch_normalization_47 (BatchN (None, 6, 6, 160) 480 ['conv2d_49[0][0]']
ormalization)
activation_41 (Activation) (None, 6, 6, 160) 0 ['batch_normalization_42[0][0]']
activation_46 (Activation) (None, 6, 6, 160) 0 ['batch_normalization_47[0][0]']
conv2d_45 (Conv2D) (None, 6, 6, 160) 179200 ['activation_41[0][0]']
conv2d_50 (Conv2D) (None, 6, 6, 160) 179200 ['activation_46[0][0]']
batch_normalization_43 (BatchN (None, 6, 6, 160) 480 ['conv2d_45[0][0]']
ormalization)
batch_normalization_48 (BatchN (None, 6, 6, 160) 480 ['conv2d_50[0][0]']
ormalization)
activation_42 (Activation) (None, 6, 6, 160) 0 ['batch_normalization_43[0][0]']
activation_47 (Activation) (None, 6, 6, 160) 0 ['batch_normalization_48[0][0]']
average_pooling2d_4 (AveragePo (None, 6, 6, 768) 0 ['mixed4[0][0]']
oling2D)
conv2d_43 (Conv2D) (None, 6, 6, 192) 147456 ['mixed4[0][0]']
conv2d_46 (Conv2D) (None, 6, 6, 192) 215040 ['activation_42[0][0]']
conv2d_51 (Conv2D) (None, 6, 6, 192) 215040 ['activation_47[0][0]']
conv2d_52 (Conv2D) (None, 6, 6, 192) 147456 ['average_pooling2d_4[0][0]']
batch_normalization_41 (BatchN (None, 6, 6, 192) 576 ['conv2d_43[0][0]']
ormalization)
batch_normalization_44 (BatchN (None, 6, 6, 192) 576 ['conv2d_46[0][0]']
ormalization)
batch_normalization_49 (BatchN (None, 6, 6, 192) 576 ['conv2d_51[0][0]']
ormalization)
batch_normalization_50 (BatchN (None, 6, 6, 192) 576 ['conv2d_52[0][0]']
ormalization)
activation_40 (Activation) (None, 6, 6, 192) 0 ['batch_normalization_41[0][0]']
activation_43 (Activation) (None, 6, 6, 192) 0 ['batch_normalization_44[0][0]']
activation_48 (Activation) (None, 6, 6, 192) 0 ['batch_normalization_49[0][0]']
activation_49 (Activation) (None, 6, 6, 192) 0 ['batch_normalization_50[0][0]']
mixed5 (Concatenate) (None, 6, 6, 768) 0 ['activation_40[0][0]',
'activation_43[0][0]',
'activation_48[0][0]',
'activation_49[0][0]']
conv2d_57 (Conv2D) (None, 6, 6, 160) 122880 ['mixed5[0][0]']
batch_normalization_55 (BatchN (None, 6, 6, 160) 480 ['conv2d_57[0][0]']
ormalization)
activation_54 (Activation) (None, 6, 6, 160) 0 ['batch_normalization_55[0][0]']
conv2d_58 (Conv2D) (None, 6, 6, 160) 179200 ['activation_54[0][0]']
batch_normalization_56 (BatchN (None, 6, 6, 160) 480 ['conv2d_58[0][0]']
ormalization)
activation_55 (Activation) (None, 6, 6, 160) 0 ['batch_normalization_56[0][0]']
conv2d_54 (Conv2D) (None, 6, 6, 160) 122880 ['mixed5[0][0]']
conv2d_59 (Conv2D) (None, 6, 6, 160) 179200 ['activation_55[0][0]']
batch_normalization_52 (BatchN (None, 6, 6, 160) 480 ['conv2d_54[0][0]']
ormalization)
batch_normalization_57 (BatchN (None, 6, 6, 160) 480 ['conv2d_59[0][0]']
ormalization)
activation_51 (Activation) (None, 6, 6, 160) 0 ['batch_normalization_52[0][0]']
activation_56 (Activation) (None, 6, 6, 160) 0 ['batch_normalization_57[0][0]']
conv2d_55 (Conv2D) (None, 6, 6, 160) 179200 ['activation_51[0][0]']
conv2d_60 (Conv2D) (None, 6, 6, 160) 179200 ['activation_56[0][0]']
batch_normalization_53 (BatchN (None, 6, 6, 160) 480 ['conv2d_55[0][0]']
ormalization)
batch_normalization_58 (BatchN (None, 6, 6, 160) 480 ['conv2d_60[0][0]']
ormalization)
activation_52 (Activation) (None, 6, 6, 160) 0 ['batch_normalization_53[0][0]']
activation_57 (Activation) (None, 6, 6, 160) 0 ['batch_normalization_58[0][0]']
average_pooling2d_5 (AveragePo (None, 6, 6, 768) 0 ['mixed5[0][0]']
oling2D)
conv2d_53 (Conv2D) (None, 6, 6, 192) 147456 ['mixed5[0][0]']
conv2d_56 (Conv2D) (None, 6, 6, 192) 215040 ['activation_52[0][0]']
conv2d_61 (Conv2D) (None, 6, 6, 192) 215040 ['activation_57[0][0]']
conv2d_62 (Conv2D) (None, 6, 6, 192) 147456 ['average_pooling2d_5[0][0]']
batch_normalization_51 (BatchN (None, 6, 6, 192) 576 ['conv2d_53[0][0]']
ormalization)
batch_normalization_54 (BatchN (None, 6, 6, 192) 576 ['conv2d_56[0][0]']
ormalization)
batch_normalization_59 (BatchN (None, 6, 6, 192) 576 ['conv2d_61[0][0]']
ormalization)
batch_normalization_60 (BatchN (None, 6, 6, 192) 576 ['conv2d_62[0][0]']
ormalization)
activation_50 (Activation) (None, 6, 6, 192) 0 ['batch_normalization_51[0][0]']
activation_53 (Activation) (None, 6, 6, 192) 0 ['batch_normalization_54[0][0]']
activation_58 (Activation) (None, 6, 6, 192) 0 ['batch_normalization_59[0][0]']
activation_59 (Activation) (None, 6, 6, 192) 0 ['batch_normalization_60[0][0]']
mixed6 (Concatenate) (None, 6, 6, 768) 0 ['activation_50[0][0]',
'activation_53[0][0]',
'activation_58[0][0]',
'activation_59[0][0]']
conv2d_67 (Conv2D) (None, 6, 6, 192) 147456 ['mixed6[0][0]']
batch_normalization_65 (BatchN (None, 6, 6, 192) 576 ['conv2d_67[0][0]']
ormalization)
activation_64 (Activation) (None, 6, 6, 192) 0 ['batch_normalization_65[0][0]']
conv2d_68 (Conv2D) (None, 6, 6, 192) 258048 ['activation_64[0][0]']
batch_normalization_66 (BatchN (None, 6, 6, 192) 576 ['conv2d_68[0][0]']
ormalization)
activation_65 (Activation) (None, 6, 6, 192) 0 ['batch_normalization_66[0][0]']
conv2d_64 (Conv2D) (None, 6, 6, 192) 147456 ['mixed6[0][0]']
conv2d_69 (Conv2D) (None, 6, 6, 192) 258048 ['activation_65[0][0]']
batch_normalization_62 (BatchN (None, 6, 6, 192) 576 ['conv2d_64[0][0]']
ormalization)
batch_normalization_67 (BatchN (None, 6, 6, 192) 576 ['conv2d_69[0][0]']
ormalization)
activation_61 (Activation) (None, 6, 6, 192) 0 ['batch_normalization_62[0][0]']
activation_66 (Activation) (None, 6, 6, 192) 0 ['batch_normalization_67[0][0]']
conv2d_65 (Conv2D) (None, 6, 6, 192) 258048 ['activation_61[0][0]']
conv2d_70 (Conv2D) (None, 6, 6, 192) 258048 ['activation_66[0][0]']
batch_normalization_63 (BatchN (None, 6, 6, 192) 576 ['conv2d_65[0][0]']
ormalization)
batch_normalization_68 (BatchN (None, 6, 6, 192) 576 ['conv2d_70[0][0]']
ormalization)
activation_62 (Activation) (None, 6, 6, 192) 0 ['batch_normalization_63[0][0]']
activation_67 (Activation) (None, 6, 6, 192) 0 ['batch_normalization_68[0][0]']
average_pooling2d_6 (AveragePo (None, 6, 6, 768) 0 ['mixed6[0][0]']
oling2D)
conv2d_63 (Conv2D) (None, 6, 6, 192) 147456 ['mixed6[0][0]']
conv2d_66 (Conv2D) (None, 6, 6, 192) 258048 ['activation_62[0][0]']
conv2d_71 (Conv2D) (None, 6, 6, 192) 258048 ['activation_67[0][0]']
conv2d_72 (Conv2D) (None, 6, 6, 192) 147456 ['average_pooling2d_6[0][0]']
batch_normalization_61 (BatchN (None, 6, 6, 192) 576 ['conv2d_63[0][0]']
ormalization)
batch_normalization_64 (BatchN (None, 6, 6, 192) 576 ['conv2d_66[0][0]']
ormalization)
batch_normalization_69 (BatchN (None, 6, 6, 192) 576 ['conv2d_71[0][0]']
ormalization)
batch_normalization_70 (BatchN (None, 6, 6, 192) 576 ['conv2d_72[0][0]']
ormalization)
activation_60 (Activation) (None, 6, 6, 192) 0 ['batch_normalization_61[0][0]']
activation_63 (Activation) (None, 6, 6, 192) 0 ['batch_normalization_64[0][0]']
activation_68 (Activation) (None, 6, 6, 192) 0 ['batch_normalization_69[0][0]']
activation_69 (Activation) (None, 6, 6, 192) 0 ['batch_normalization_70[0][0]']
mixed7 (Concatenate) (None, 6, 6, 768) 0 ['activation_60[0][0]',
'activation_63[0][0]',
'activation_68[0][0]',
'activation_69[0][0]']
conv2d_75 (Conv2D) (None, 6, 6, 192) 147456 ['mixed7[0][0]']
batch_normalization_73 (BatchN (None, 6, 6, 192) 576 ['conv2d_75[0][0]']
ormalization)
activation_72 (Activation) (None, 6, 6, 192) 0 ['batch_normalization_73[0][0]']
conv2d_76 (Conv2D) (None, 6, 6, 192) 258048 ['activation_72[0][0]']
batch_normalization_74 (BatchN (None, 6, 6, 192) 576 ['conv2d_76[0][0]']
ormalization)
activation_73 (Activation) (None, 6, 6, 192) 0 ['batch_normalization_74[0][0]']
conv2d_73 (Conv2D) (None, 6, 6, 192) 147456 ['mixed7[0][0]']
conv2d_77 (Conv2D) (None, 6, 6, 192) 258048 ['activation_73[0][0]']
batch_normalization_71 (BatchN (None, 6, 6, 192) 576 ['conv2d_73[0][0]']
ormalization)
batch_normalization_75 (BatchN (None, 6, 6, 192) 576 ['conv2d_77[0][0]']
ormalization)
activation_70 (Activation) (None, 6, 6, 192) 0 ['batch_normalization_71[0][0]']
activation_74 (Activation) (None, 6, 6, 192) 0 ['batch_normalization_75[0][0]']
conv2d_74 (Conv2D) (None, 2, 2, 320) 552960 ['activation_70[0][0]']
conv2d_78 (Conv2D) (None, 2, 2, 192) 331776 ['activation_74[0][0]']
batch_normalization_72 (BatchN (None, 2, 2, 320) 960 ['conv2d_74[0][0]']
ormalization)
batch_normalization_76 (BatchN (None, 2, 2, 192) 576 ['conv2d_78[0][0]']
ormalization)
activation_71 (Activation) (None, 2, 2, 320) 0 ['batch_normalization_72[0][0]']
activation_75 (Activation) (None, 2, 2, 192) 0 ['batch_normalization_76[0][0]']
max_pooling2d_7 (MaxPooling2D) (None, 2, 2, 768) 0 ['mixed7[0][0]']
mixed8 (Concatenate) (None, 2, 2, 1280) 0 ['activation_71[0][0]',
'activation_75[0][0]',
'max_pooling2d_7[0][0]']
conv2d_83 (Conv2D) (None, 2, 2, 448) 573440 ['mixed8[0][0]']
batch_normalization_81 (BatchN (None, 2, 2, 448) 1344 ['conv2d_83[0][0]']
ormalization)
activation_80 (Activation) (None, 2, 2, 448) 0 ['batch_normalization_81[0][0]']
conv2d_80 (Conv2D) (None, 2, 2, 384) 491520 ['mixed8[0][0]']
conv2d_84 (Conv2D) (None, 2, 2, 384) 1548288 ['activation_80[0][0]']
batch_normalization_78 (BatchN (None, 2, 2, 384) 1152 ['conv2d_80[0][0]']
ormalization)
batch_normalization_82 (BatchN (None, 2, 2, 384) 1152 ['conv2d_84[0][0]']
ormalization)
activation_77 (Activation) (None, 2, 2, 384) 0 ['batch_normalization_78[0][0]']
activation_81 (Activation) (None, 2, 2, 384) 0 ['batch_normalization_82[0][0]']
conv2d_81 (Conv2D) (None, 2, 2, 384) 442368 ['activation_77[0][0]']
conv2d_82 (Conv2D) (None, 2, 2, 384) 442368 ['activation_77[0][0]']
conv2d_85 (Conv2D) (None, 2, 2, 384) 442368 ['activation_81[0][0]']
conv2d_86 (Conv2D) (None, 2, 2, 384) 442368 ['activation_81[0][0]']
average_pooling2d_7 (AveragePo (None, 2, 2, 1280) 0 ['mixed8[0][0]']
oling2D)
conv2d_79 (Conv2D) (None, 2, 2, 320) 409600 ['mixed8[0][0]']
batch_normalization_79 (BatchN (None, 2, 2, 384) 1152 ['conv2d_81[0][0]']
ormalization)
batch_normalization_80 (BatchN (None, 2, 2, 384) 1152 ['conv2d_82[0][0]']
ormalization)
batch_normalization_83 (BatchN (None, 2, 2, 384) 1152 ['conv2d_85[0][0]']
ormalization)
batch_normalization_84 (BatchN (None, 2, 2, 384) 1152 ['conv2d_86[0][0]']
ormalization)
conv2d_87 (Conv2D) (None, 2, 2, 192) 245760 ['average_pooling2d_7[0][0]']
batch_normalization_77 (BatchN (None, 2, 2, 320) 960 ['conv2d_79[0][0]']
ormalization)
activation_78 (Activation) (None, 2, 2, 384) 0 ['batch_normalization_79[0][0]']
activation_79 (Activation) (None, 2, 2, 384) 0 ['batch_normalization_80[0][0]']
activation_82 (Activation) (None, 2, 2, 384) 0 ['batch_normalization_83[0][0]']
activation_83 (Activation) (None, 2, 2, 384) 0 ['batch_normalization_84[0][0]']
batch_normalization_85 (BatchN (None, 2, 2, 192) 576 ['conv2d_87[0][0]']
ormalization)
activation_76 (Activation) (None, 2, 2, 320) 0 ['batch_normalization_77[0][0]']
mixed9_0 (Concatenate) (None, 2, 2, 768) 0 ['activation_78[0][0]',
'activation_79[0][0]']
concatenate (Concatenate) (None, 2, 2, 768) 0 ['activation_82[0][0]',
'activation_83[0][0]']
activation_84 (Activation) (None, 2, 2, 192) 0 ['batch_normalization_85[0][0]']
mixed9 (Concatenate) (None, 2, 2, 2048) 0 ['activation_76[0][0]',
'mixed9_0[0][0]',
'concatenate[0][0]',
'activation_84[0][0]']
conv2d_92 (Conv2D) (None, 2, 2, 448) 917504 ['mixed9[0][0]']
batch_normalization_90 (BatchN (None, 2, 2, 448) 1344 ['conv2d_92[0][0]']
ormalization)
activation_89 (Activation) (None, 2, 2, 448) 0 ['batch_normalization_90[0][0]']
conv2d_89 (Conv2D) (None, 2, 2, 384) 786432 ['mixed9[0][0]']
conv2d_93 (Conv2D) (None, 2, 2, 384) 1548288 ['activation_89[0][0]']
batch_normalization_87 (BatchN (None, 2, 2, 384) 1152 ['conv2d_89[0][0]']
ormalization)
batch_normalization_91 (BatchN (None, 2, 2, 384) 1152 ['conv2d_93[0][0]']
ormalization)
activation_86 (Activation) (None, 2, 2, 384) 0 ['batch_normalization_87[0][0]']
activation_90 (Activation) (None, 2, 2, 384) 0 ['batch_normalization_91[0][0]']
conv2d_90 (Conv2D) (None, 2, 2, 384) 442368 ['activation_86[0][0]']
conv2d_91 (Conv2D) (None, 2, 2, 384) 442368 ['activation_86[0][0]']
conv2d_94 (Conv2D) (None, 2, 2, 384) 442368 ['activation_90[0][0]']
conv2d_95 (Conv2D) (None, 2, 2, 384) 442368 ['activation_90[0][0]']
average_pooling2d_8 (AveragePo (None, 2, 2, 2048) 0 ['mixed9[0][0]']
oling2D)
conv2d_88 (Conv2D) (None, 2, 2, 320) 655360 ['mixed9[0][0]']
batch_normalization_88 (BatchN (None, 2, 2, 384) 1152 ['conv2d_90[0][0]']
ormalization)
batch_normalization_89 (BatchN (None, 2, 2, 384) 1152 ['conv2d_91[0][0]']
ormalization)
batch_normalization_92 (BatchN (None, 2, 2, 384) 1152 ['conv2d_94[0][0]']
ormalization)
batch_normalization_93 (BatchN (None, 2, 2, 384) 1152 ['conv2d_95[0][0]']
ormalization)
conv2d_96 (Conv2D) (None, 2, 2, 192) 393216 ['average_pooling2d_8[0][0]']
batch_normalization_86 (BatchN (None, 2, 2, 320) 960 ['conv2d_88[0][0]']
ormalization)
activation_87 (Activation) (None, 2, 2, 384) 0 ['batch_normalization_88[0][0]']
activation_88 (Activation) (None, 2, 2, 384) 0 ['batch_normalization_89[0][0]']
activation_91 (Activation) (None, 2, 2, 384) 0 ['batch_normalization_92[0][0]']
activation_92 (Activation) (None, 2, 2, 384) 0 ['batch_normalization_93[0][0]']
batch_normalization_94 (BatchN (None, 2, 2, 192) 576 ['conv2d_96[0][0]']
ormalization)
activation_85 (Activation) (None, 2, 2, 320) 0 ['batch_normalization_86[0][0]']
mixed9_1 (Concatenate) (None, 2, 2, 768) 0 ['activation_87[0][0]',
'activation_88[0][0]']
concatenate_1 (Concatenate) (None, 2, 2, 768) 0 ['activation_91[0][0]',
'activation_92[0][0]']
activation_93 (Activation) (None, 2, 2, 192) 0 ['batch_normalization_94[0][0]']
mixed10 (Concatenate) (None, 2, 2, 2048) 0 ['activation_85[0][0]',
'mixed9_1[0][0]',
'concatenate_1[0][0]',
'activation_93[0][0]']
flatten_2 (Flatten) (None, 8192) 0 ['mixed10[0][0]']
dense_5 (Dense) (None, 1024) 8389632 ['flatten_2[0][0]']
dropout_5 (Dropout) (None, 1024) 0 ['dense_5[0][0]']
dense_6 (Dense) (None, 2) 2050 ['dropout_5[0][0]']
==================================================================================================
Total params: 30,194,466
Trainable params: 8,391,682
Non-trainable params: 21,802,784
__________________________________________________________________________________________________
# Plot the model
tf.keras.utils.plot_model(model_clf_3, show_shapes=True)
# Fit the model
history_clf_3 = model_clf_3.fit(Train_X_clf_3, Train_y, epochs=10, batch_size=32, validation_split=0.2, callbacks=[early_stop, reduce_lr])
Epoch 1/10 125/125 [==============================] - 11s 54ms/step - loss: 0.8575 - accuracy: 0.7735 - val_loss: 0.4386 - val_accuracy: 0.7970 - lr: 1.0000e-04 Epoch 2/10 125/125 [==============================] - 5s 41ms/step - loss: 0.3932 - accuracy: 0.8422 - val_loss: 0.4119 - val_accuracy: 0.8470 - lr: 1.0000e-04 Epoch 3/10 125/125 [==============================] - 5s 41ms/step - loss: 0.3293 - accuracy: 0.8610 - val_loss: 0.4521 - val_accuracy: 0.8370 - lr: 1.0000e-04 Epoch 4/10 125/125 [==============================] - 5s 38ms/step - loss: 0.2718 - accuracy: 0.8880 - val_loss: 0.4311 - val_accuracy: 0.8100 - lr: 1.0000e-04 Epoch 5/10 125/125 [==============================] - 5s 38ms/step - loss: 0.2147 - accuracy: 0.9165 - val_loss: 0.4225 - val_accuracy: 0.8250 - lr: 1.0000e-04 Epoch 6/10 125/125 [==============================] - 5s 38ms/step - loss: 0.1924 - accuracy: 0.9230 - val_loss: 0.4489 - val_accuracy: 0.8220 - lr: 1.0000e-04 Epoch 7/10 125/125 [==============================] - 5s 38ms/step - loss: 0.1534 - accuracy: 0.9367 - val_loss: 0.4705 - val_accuracy: 0.8470 - lr: 1.0000e-04 Epoch 8/10 125/125 [==============================] - 5s 38ms/step - loss: 0.1184 - accuracy: 0.9585 - val_loss: 0.4888 - val_accuracy: 0.8320 - lr: 1.0000e-04 Epoch 9/10 125/125 [==============================] - 5s 38ms/step - loss: 0.0983 - accuracy: 0.9657 - val_loss: 0.5093 - val_accuracy: 0.8250 - lr: 1.0000e-04 Epoch 10/10 125/125 [==============================] - 5s 38ms/step - loss: 0.0728 - accuracy: 0.9795 - val_loss: 0.4997 - val_accuracy: 0.8220 - lr: 1.0000e-04
# Plot accuracy and loss
plot_accuracy_loss(history=history_clf_3, model_type='CLF')
# Model Evaluation
eval_acc_clf_3, eval_loss_clf_3, train_acc_clf_3, train_loss_clf_3 = evaluate_model(model_clf_3, Eval_X_clf_3, Eval_y, Train_X_clf_3, Train_y, 'CLF')
57/57 [==============================] - 2s 41ms/step - loss: 0.4629 - accuracy: 0.8335 157/157 [==============================] - 5s 30ms/step - loss: 0.1337 - accuracy: 0.9612 Accuracy on Eval set = 83.35% & Loss on Eval set = 0.4629
# Classification report & Confusion matrix
clf_report_and_conf_matrix(model_clf_3, Eval_X_clf_3, Eval_y)
Classification Report:
precision recall f1-score support
0 0.82 0.85 0.84 895
1 0.85 0.81 0.83 901
accuracy 0.83 1796
macro avg 0.83 0.83 0.83 1796
weighted avg 0.83 0.83 0.83 1796
# Append the scores & loss
clf_model_scores['Model'].append('InceptionV3')
clf_model_scores['Train Score'].append(train_acc_clf_3)
clf_model_scores['Eval Score'].append(eval_acc_clf_3)
clf_model_scores['Train Loss'].append(train_loss_clf_3)
clf_model_scores['Eval Loss'].append(eval_loss_clf_3)
del Train_X_clf_3
del Eval_X_clf_3
# Convert dictionary into DataFrame
clf_models_df = pd.DataFrame(clf_model_scores)
clf_models_df
| Model | Train Score | Eval Score | Train Loss | Eval Loss | |
|---|---|---|---|---|---|
| 0 | Basic CNN | 82.72 | 83.24 | 0.4027 | 0.4106 |
| 1 | MobileNet | 92.84 | 86.86 | 9.3639 | 9.5320 |
| 2 | InceptionV3 | 96.12 | 83.35 | 0.1337 | 0.4629 |
# Plot the accuracy and loss for all classifier models
plt.figure(figsize=(20,6))
plt.subplot(1, 2, 1)
sns.lineplot(x=clf_models_df.Model, y=clf_models_df['Train Score'], marker='o', label='Train Score')
sns.lineplot(x=clf_models_df.Model, y=clf_models_df['Eval Score'], marker='o', label='Eval Score', linestyle='--')
plt.xlabel('Models')
plt.ylabel('Accuracy')
plt.title('Model Accuracy')
plt.subplot(1, 2, 2)
sns.lineplot(x=clf_models_df.Model, y=clf_models_df['Train Loss'], marker='s', label='Train Loss')
sns.lineplot(x=clf_models_df.Model, y=clf_models_df['Eval Loss'], marker='s', label='Eval Loss', linestyle='--')
plt.xlabel('Models')
plt.ylabel('Loss')
plt.title('Model Loss')
plt.show()
# Apply Pre-processing
Train_X_unet_1 = tf.keras.applications.mobilenet.preprocess_input(np.copy(Train_X_unet))
Eval_X_unet_1 = tf.keras.applications.mobilenet.preprocess_input(np.copy(Eval_X_unet))
# Instantiates the MobileNet architecture
model_unet_1_temp = tf.keras.applications.mobilenet.MobileNet(input_shape=(128, 128, 3), alpha=1.0, include_top=False, weights='imagenet')
# Previous layers are non trainable
for layer in model_unet_1_temp.layers:
layer.trainable=False
block0 = model_unet_1_temp.input
block1 = model_unet_1_temp.get_layer('conv_pw_1_relu').output
block2 = model_unet_1_temp.get_layer('conv_pw_3_relu').output
block3 = model_unet_1_temp.get_layer('conv_pw_5_relu').output
block4 = model_unet_1_temp.get_layer('conv_pw_11_relu').output
block5 = model_unet_1_temp.get_layer('conv_pw_13_relu').output
# Decoder Blocks
x = tf.keras.layers.Concatenate()([tf.keras.layers.UpSampling2D()(block5), block4])
x = tf.keras.layers.Concatenate()([tf.keras.layers.UpSampling2D()(x), block3])
x = tf.keras.layers.Concatenate()([tf.keras.layers.UpSampling2D()(x), block2])
x = tf.keras.layers.Concatenate()([tf.keras.layers.UpSampling2D()(x), block1])
x = tf.keras.layers.Concatenate()([tf.keras.layers.UpSampling2D()(x), block0])
x = tf.keras.layers.LeakyReLU(alpha=0.1)(x)
x = tf.keras.layers.Conv2D(16, kernel_size=3, activation='relu', padding='same')(x)
x = tf.keras.layers.Dropout(0.3)(x)
x = tf.keras.layers.Conv2D(1, kernel_size=1, activation='sigmoid')(x)
x = tf.keras.layers.Reshape(target_shape=(128,128))(x)
model_unet_1 = tf.keras.Model(inputs=model_unet_1_temp.input, outputs=x)
# Model summary
model_unet_1.summary()
# Define optimizer and compile the model
model_unet_1.compile(
optimizer = tf.keras.optimizers.Adam(learning_rate=0.0001, beta_1=0.9, beta_2=0.999, epsilon=1e-07),
loss = loss,
metrics = [dice_coefficient])
Model: "model_2"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_3 (InputLayer) [(None, 128, 128, 3 0 []
)]
conv1 (Conv2D) (None, 64, 64, 32) 864 ['input_3[0][0]']
conv1_bn (BatchNormalization) (None, 64, 64, 32) 128 ['conv1[0][0]']
conv1_relu (ReLU) (None, 64, 64, 32) 0 ['conv1_bn[0][0]']
conv_dw_1 (DepthwiseConv2D) (None, 64, 64, 32) 288 ['conv1_relu[0][0]']
conv_dw_1_bn (BatchNormalizati (None, 64, 64, 32) 128 ['conv_dw_1[0][0]']
on)
conv_dw_1_relu (ReLU) (None, 64, 64, 32) 0 ['conv_dw_1_bn[0][0]']
conv_pw_1 (Conv2D) (None, 64, 64, 64) 2048 ['conv_dw_1_relu[0][0]']
conv_pw_1_bn (BatchNormalizati (None, 64, 64, 64) 256 ['conv_pw_1[0][0]']
on)
conv_pw_1_relu (ReLU) (None, 64, 64, 64) 0 ['conv_pw_1_bn[0][0]']
conv_pad_2 (ZeroPadding2D) (None, 65, 65, 64) 0 ['conv_pw_1_relu[0][0]']
conv_dw_2 (DepthwiseConv2D) (None, 32, 32, 64) 576 ['conv_pad_2[0][0]']
conv_dw_2_bn (BatchNormalizati (None, 32, 32, 64) 256 ['conv_dw_2[0][0]']
on)
conv_dw_2_relu (ReLU) (None, 32, 32, 64) 0 ['conv_dw_2_bn[0][0]']
conv_pw_2 (Conv2D) (None, 32, 32, 128) 8192 ['conv_dw_2_relu[0][0]']
conv_pw_2_bn (BatchNormalizati (None, 32, 32, 128) 512 ['conv_pw_2[0][0]']
on)
conv_pw_2_relu (ReLU) (None, 32, 32, 128) 0 ['conv_pw_2_bn[0][0]']
conv_dw_3 (DepthwiseConv2D) (None, 32, 32, 128) 1152 ['conv_pw_2_relu[0][0]']
conv_dw_3_bn (BatchNormalizati (None, 32, 32, 128) 512 ['conv_dw_3[0][0]']
on)
conv_dw_3_relu (ReLU) (None, 32, 32, 128) 0 ['conv_dw_3_bn[0][0]']
conv_pw_3 (Conv2D) (None, 32, 32, 128) 16384 ['conv_dw_3_relu[0][0]']
conv_pw_3_bn (BatchNormalizati (None, 32, 32, 128) 512 ['conv_pw_3[0][0]']
on)
conv_pw_3_relu (ReLU) (None, 32, 32, 128) 0 ['conv_pw_3_bn[0][0]']
conv_pad_4 (ZeroPadding2D) (None, 33, 33, 128) 0 ['conv_pw_3_relu[0][0]']
conv_dw_4 (DepthwiseConv2D) (None, 16, 16, 128) 1152 ['conv_pad_4[0][0]']
conv_dw_4_bn (BatchNormalizati (None, 16, 16, 128) 512 ['conv_dw_4[0][0]']
on)
conv_dw_4_relu (ReLU) (None, 16, 16, 128) 0 ['conv_dw_4_bn[0][0]']
conv_pw_4 (Conv2D) (None, 16, 16, 256) 32768 ['conv_dw_4_relu[0][0]']
conv_pw_4_bn (BatchNormalizati (None, 16, 16, 256) 1024 ['conv_pw_4[0][0]']
on)
conv_pw_4_relu (ReLU) (None, 16, 16, 256) 0 ['conv_pw_4_bn[0][0]']
conv_dw_5 (DepthwiseConv2D) (None, 16, 16, 256) 2304 ['conv_pw_4_relu[0][0]']
conv_dw_5_bn (BatchNormalizati (None, 16, 16, 256) 1024 ['conv_dw_5[0][0]']
on)
conv_dw_5_relu (ReLU) (None, 16, 16, 256) 0 ['conv_dw_5_bn[0][0]']
conv_pw_5 (Conv2D) (None, 16, 16, 256) 65536 ['conv_dw_5_relu[0][0]']
conv_pw_5_bn (BatchNormalizati (None, 16, 16, 256) 1024 ['conv_pw_5[0][0]']
on)
conv_pw_5_relu (ReLU) (None, 16, 16, 256) 0 ['conv_pw_5_bn[0][0]']
conv_pad_6 (ZeroPadding2D) (None, 17, 17, 256) 0 ['conv_pw_5_relu[0][0]']
conv_dw_6 (DepthwiseConv2D) (None, 8, 8, 256) 2304 ['conv_pad_6[0][0]']
conv_dw_6_bn (BatchNormalizati (None, 8, 8, 256) 1024 ['conv_dw_6[0][0]']
on)
conv_dw_6_relu (ReLU) (None, 8, 8, 256) 0 ['conv_dw_6_bn[0][0]']
conv_pw_6 (Conv2D) (None, 8, 8, 512) 131072 ['conv_dw_6_relu[0][0]']
conv_pw_6_bn (BatchNormalizati (None, 8, 8, 512) 2048 ['conv_pw_6[0][0]']
on)
conv_pw_6_relu (ReLU) (None, 8, 8, 512) 0 ['conv_pw_6_bn[0][0]']
conv_dw_7 (DepthwiseConv2D) (None, 8, 8, 512) 4608 ['conv_pw_6_relu[0][0]']
conv_dw_7_bn (BatchNormalizati (None, 8, 8, 512) 2048 ['conv_dw_7[0][0]']
on)
conv_dw_7_relu (ReLU) (None, 8, 8, 512) 0 ['conv_dw_7_bn[0][0]']
conv_pw_7 (Conv2D) (None, 8, 8, 512) 262144 ['conv_dw_7_relu[0][0]']
conv_pw_7_bn (BatchNormalizati (None, 8, 8, 512) 2048 ['conv_pw_7[0][0]']
on)
conv_pw_7_relu (ReLU) (None, 8, 8, 512) 0 ['conv_pw_7_bn[0][0]']
conv_dw_8 (DepthwiseConv2D) (None, 8, 8, 512) 4608 ['conv_pw_7_relu[0][0]']
conv_dw_8_bn (BatchNormalizati (None, 8, 8, 512) 2048 ['conv_dw_8[0][0]']
on)
conv_dw_8_relu (ReLU) (None, 8, 8, 512) 0 ['conv_dw_8_bn[0][0]']
conv_pw_8 (Conv2D) (None, 8, 8, 512) 262144 ['conv_dw_8_relu[0][0]']
conv_pw_8_bn (BatchNormalizati (None, 8, 8, 512) 2048 ['conv_pw_8[0][0]']
on)
conv_pw_8_relu (ReLU) (None, 8, 8, 512) 0 ['conv_pw_8_bn[0][0]']
conv_dw_9 (DepthwiseConv2D) (None, 8, 8, 512) 4608 ['conv_pw_8_relu[0][0]']
conv_dw_9_bn (BatchNormalizati (None, 8, 8, 512) 2048 ['conv_dw_9[0][0]']
on)
conv_dw_9_relu (ReLU) (None, 8, 8, 512) 0 ['conv_dw_9_bn[0][0]']
conv_pw_9 (Conv2D) (None, 8, 8, 512) 262144 ['conv_dw_9_relu[0][0]']
conv_pw_9_bn (BatchNormalizati (None, 8, 8, 512) 2048 ['conv_pw_9[0][0]']
on)
conv_pw_9_relu (ReLU) (None, 8, 8, 512) 0 ['conv_pw_9_bn[0][0]']
conv_dw_10 (DepthwiseConv2D) (None, 8, 8, 512) 4608 ['conv_pw_9_relu[0][0]']
conv_dw_10_bn (BatchNormalizat (None, 8, 8, 512) 2048 ['conv_dw_10[0][0]']
ion)
conv_dw_10_relu (ReLU) (None, 8, 8, 512) 0 ['conv_dw_10_bn[0][0]']
conv_pw_10 (Conv2D) (None, 8, 8, 512) 262144 ['conv_dw_10_relu[0][0]']
conv_pw_10_bn (BatchNormalizat (None, 8, 8, 512) 2048 ['conv_pw_10[0][0]']
ion)
conv_pw_10_relu (ReLU) (None, 8, 8, 512) 0 ['conv_pw_10_bn[0][0]']
conv_dw_11 (DepthwiseConv2D) (None, 8, 8, 512) 4608 ['conv_pw_10_relu[0][0]']
conv_dw_11_bn (BatchNormalizat (None, 8, 8, 512) 2048 ['conv_dw_11[0][0]']
ion)
conv_dw_11_relu (ReLU) (None, 8, 8, 512) 0 ['conv_dw_11_bn[0][0]']
conv_pw_11 (Conv2D) (None, 8, 8, 512) 262144 ['conv_dw_11_relu[0][0]']
conv_pw_11_bn (BatchNormalizat (None, 8, 8, 512) 2048 ['conv_pw_11[0][0]']
ion)
conv_pw_11_relu (ReLU) (None, 8, 8, 512) 0 ['conv_pw_11_bn[0][0]']
conv_pad_12 (ZeroPadding2D) (None, 9, 9, 512) 0 ['conv_pw_11_relu[0][0]']
conv_dw_12 (DepthwiseConv2D) (None, 4, 4, 512) 4608 ['conv_pad_12[0][0]']
conv_dw_12_bn (BatchNormalizat (None, 4, 4, 512) 2048 ['conv_dw_12[0][0]']
ion)
conv_dw_12_relu (ReLU) (None, 4, 4, 512) 0 ['conv_dw_12_bn[0][0]']
conv_pw_12 (Conv2D) (None, 4, 4, 1024) 524288 ['conv_dw_12_relu[0][0]']
conv_pw_12_bn (BatchNormalizat (None, 4, 4, 1024) 4096 ['conv_pw_12[0][0]']
ion)
conv_pw_12_relu (ReLU) (None, 4, 4, 1024) 0 ['conv_pw_12_bn[0][0]']
conv_dw_13 (DepthwiseConv2D) (None, 4, 4, 1024) 9216 ['conv_pw_12_relu[0][0]']
conv_dw_13_bn (BatchNormalizat (None, 4, 4, 1024) 4096 ['conv_dw_13[0][0]']
ion)
conv_dw_13_relu (ReLU) (None, 4, 4, 1024) 0 ['conv_dw_13_bn[0][0]']
conv_pw_13 (Conv2D) (None, 4, 4, 1024) 1048576 ['conv_dw_13_relu[0][0]']
conv_pw_13_bn (BatchNormalizat (None, 4, 4, 1024) 4096 ['conv_pw_13[0][0]']
ion)
conv_pw_13_relu (ReLU) (None, 4, 4, 1024) 0 ['conv_pw_13_bn[0][0]']
up_sampling2d (UpSampling2D) (None, 8, 8, 1024) 0 ['conv_pw_13_relu[0][0]']
concatenate_2 (Concatenate) (None, 8, 8, 1536) 0 ['up_sampling2d[0][0]',
'conv_pw_11_relu[0][0]']
up_sampling2d_1 (UpSampling2D) (None, 16, 16, 1536 0 ['concatenate_2[0][0]']
)
concatenate_3 (Concatenate) (None, 16, 16, 1792 0 ['up_sampling2d_1[0][0]',
) 'conv_pw_5_relu[0][0]']
up_sampling2d_2 (UpSampling2D) (None, 32, 32, 1792 0 ['concatenate_3[0][0]']
)
concatenate_4 (Concatenate) (None, 32, 32, 1920 0 ['up_sampling2d_2[0][0]',
) 'conv_pw_3_relu[0][0]']
up_sampling2d_3 (UpSampling2D) (None, 64, 64, 1920 0 ['concatenate_4[0][0]']
)
concatenate_5 (Concatenate) (None, 64, 64, 1984 0 ['up_sampling2d_3[0][0]',
) 'conv_pw_1_relu[0][0]']
up_sampling2d_4 (UpSampling2D) (None, 128, 128, 19 0 ['concatenate_5[0][0]']
84)
concatenate_6 (Concatenate) (None, 128, 128, 19 0 ['up_sampling2d_4[0][0]',
87) 'input_3[0][0]']
leaky_re_lu (LeakyReLU) (None, 128, 128, 19 0 ['concatenate_6[0][0]']
87)
conv2d_97 (Conv2D) (None, 128, 128, 16 286144 ['leaky_re_lu[0][0]']
)
dropout_6 (Dropout) (None, 128, 128, 16 0 ['conv2d_97[0][0]']
)
conv2d_98 (Conv2D) (None, 128, 128, 1) 17 ['dropout_6[0][0]']
reshape (Reshape) (None, 128, 128) 0 ['conv2d_98[0][0]']
==================================================================================================
Total params: 3,515,025
Trainable params: 286,161
Non-trainable params: 3,228,864
__________________________________________________________________________________________________
# Plot the model
tf.keras.utils.plot_model(model_unet_1, show_shapes=True)
# Fit the model
history_unet_1 = model_unet_1.fit(Train_X_unet_1, Train_mask_unet, epochs=20, batch_size=32, validation_split=0.35, callbacks=[early_stop, reduce_lr])
Epoch 1/20 69/69 [==============================] - 106s 1s/step - loss: 1.2717 - dice_coefficient: 0.4543 - val_loss: 0.8655 - val_dice_coefficient: 0.5723 - lr: 1.0000e-04 Epoch 2/20 69/69 [==============================] - 89s 1s/step - loss: 0.9144 - dice_coefficient: 0.5612 - val_loss: 0.7816 - val_dice_coefficient: 0.6005 - lr: 1.0000e-04 Epoch 3/20 69/69 [==============================] - 90s 1s/step - loss: 0.8530 - dice_coefficient: 0.5857 - val_loss: 0.7570 - val_dice_coefficient: 0.6142 - lr: 1.0000e-04 Epoch 4/20 69/69 [==============================] - 90s 1s/step - loss: 0.8234 - dice_coefficient: 0.5978 - val_loss: 0.7436 - val_dice_coefficient: 0.6169 - lr: 1.0000e-04 Epoch 5/20 69/69 [==============================] - 90s 1s/step - loss: 0.8024 - dice_coefficient: 0.6051 - val_loss: 0.7317 - val_dice_coefficient: 0.6269 - lr: 1.0000e-04 Epoch 6/20 69/69 [==============================] - 90s 1s/step - loss: 0.7847 - dice_coefficient: 0.6122 - val_loss: 0.7234 - val_dice_coefficient: 0.6373 - lr: 1.0000e-04 Epoch 7/20 69/69 [==============================] - 90s 1s/step - loss: 0.7737 - dice_coefficient: 0.6166 - val_loss: 0.7244 - val_dice_coefficient: 0.6272 - lr: 1.0000e-04 Epoch 8/20 69/69 [==============================] - 90s 1s/step - loss: 0.7611 - dice_coefficient: 0.6214 - val_loss: 0.7193 - val_dice_coefficient: 0.6308 - lr: 1.0000e-04 Epoch 9/20 69/69 [==============================] - 90s 1s/step - loss: 0.7516 - dice_coefficient: 0.6249 - val_loss: 0.7225 - val_dice_coefficient: 0.6312 - lr: 1.0000e-04 Epoch 10/20 69/69 [==============================] - 90s 1s/step - loss: 0.7486 - dice_coefficient: 0.6266 - val_loss: 0.7125 - val_dice_coefficient: 0.6393 - lr: 1.0000e-04 Epoch 11/20 69/69 [==============================] - 90s 1s/step - loss: 0.7384 - dice_coefficient: 0.6306 - val_loss: 0.7142 - val_dice_coefficient: 0.6281 - lr: 1.0000e-04 Epoch 12/20 69/69 [==============================] - 90s 1s/step - loss: 0.7380 - dice_coefficient: 0.6305 - val_loss: 0.7223 - val_dice_coefficient: 0.6285 - lr: 1.0000e-04 Epoch 13/20 69/69 [==============================] - 90s 1s/step - loss: 0.7267 - dice_coefficient: 0.6351 - val_loss: 0.7089 - val_dice_coefficient: 0.6334 - lr: 1.0000e-04 Epoch 14/20 69/69 [==============================] - 90s 1s/step - loss: 0.7332 - dice_coefficient: 0.6331 - val_loss: 0.7215 - val_dice_coefficient: 0.6395 - lr: 1.0000e-04 Epoch 15/20 69/69 [==============================] - 90s 1s/step - loss: 0.7218 - dice_coefficient: 0.6377 - val_loss: 0.7046 - val_dice_coefficient: 0.6320 - lr: 1.0000e-04 Epoch 16/20 69/69 [==============================] - 90s 1s/step - loss: 0.7176 - dice_coefficient: 0.6384 - val_loss: 0.7239 - val_dice_coefficient: 0.6418 - lr: 1.0000e-04 Epoch 17/20 69/69 [==============================] - 90s 1s/step - loss: 0.7131 - dice_coefficient: 0.6408 - val_loss: 0.7067 - val_dice_coefficient: 0.6349 - lr: 1.0000e-04 Epoch 18/20 69/69 [==============================] - 90s 1s/step - loss: 0.7049 - dice_coefficient: 0.6435 - val_loss: 0.7322 - val_dice_coefficient: 0.6361 - lr: 1.0000e-04 Epoch 19/20 69/69 [==============================] - 90s 1s/step - loss: 0.7050 - dice_coefficient: 0.6436 - val_loss: 0.7126 - val_dice_coefficient: 0.6452 - lr: 1.0000e-04 Epoch 20/20 69/69 [==============================] - 90s 1s/step - loss: 0.7011 - dice_coefficient: 0.6450 - val_loss: 0.6997 - val_dice_coefficient: 0.6386 - lr: 1.0000e-04
# Plot accuracy and loss
plot_accuracy_loss(history=history_unet_1, model_type='UNET')
# Model Evaluation
eval_acc_unet_1, eval_loss_unet_1, train_acc_unet_1, train_loss_unet_1 = evaluate_model(model_unet_1, Eval_X_unet_1, Eval_mask_unet, Train_X_unet_1, Train_mask_unet, 'UNET')
1/1 [==============================] - 2s 2s/step - loss: 0.6187 - dice_coefficient: 0.6747 106/106 [==============================] - 44s 420ms/step - loss: 0.6790 - dice_coefficient: 0.6470 Dice Coefficient on Eval set = 0.6747 & Loss on Eval set = 0.6187
# Append the scores & loss
unet_model_scores['Model'].append('MobileNet')
unet_model_scores['Train Score'].append(train_acc_unet_1)
unet_model_scores['Eval Score'].append(eval_acc_unet_1)
unet_model_scores['Train Loss'].append(train_loss_unet_1)
unet_model_scores['Eval Loss'].append(eval_loss_unet_1)
del Train_X_unet_1
del Eval_X_unet_1
!pip install segmentation_models
import segmentation_models
segmentation_models.set_framework('tf.keras')
from segmentation_models import Unet
from segmentation_models import get_preprocessing
Collecting segmentation_models
Downloading segmentation_models-1.0.1-py3-none-any.whl (33 kB)
Collecting image-classifiers==1.0.0
Downloading image_classifiers-1.0.0-py3-none-any.whl (19 kB)
Collecting efficientnet==1.0.0
Downloading efficientnet-1.0.0-py3-none-any.whl (17 kB)
Collecting keras-applications<=1.0.8,>=1.0.7
Downloading Keras_Applications-1.0.8-py3-none-any.whl (50 kB)
|████████████████████████████████| 50 kB 7.2 MB/s
Requirement already satisfied: scikit-image in /usr/local/lib/python3.7/dist-packages (from efficientnet==1.0.0->segmentation_models) (0.18.3)
Requirement already satisfied: numpy>=1.9.1 in /usr/local/lib/python3.7/dist-packages (from keras-applications<=1.0.8,>=1.0.7->segmentation_models) (1.21.6)
Requirement already satisfied: h5py in /usr/local/lib/python3.7/dist-packages (from keras-applications<=1.0.8,>=1.0.7->segmentation_models) (3.1.0)
Requirement already satisfied: cached-property in /usr/local/lib/python3.7/dist-packages (from h5py->keras-applications<=1.0.8,>=1.0.7->segmentation_models) (1.5.2)
Requirement already satisfied: imageio>=2.3.0 in /usr/local/lib/python3.7/dist-packages (from scikit-image->efficientnet==1.0.0->segmentation_models) (2.4.1)
Requirement already satisfied: PyWavelets>=1.1.1 in /usr/local/lib/python3.7/dist-packages (from scikit-image->efficientnet==1.0.0->segmentation_models) (1.3.0)
Requirement already satisfied: networkx>=2.0 in /usr/local/lib/python3.7/dist-packages (from scikit-image->efficientnet==1.0.0->segmentation_models) (2.6.3)
Requirement already satisfied: tifffile>=2019.7.26 in /usr/local/lib/python3.7/dist-packages (from scikit-image->efficientnet==1.0.0->segmentation_models) (2021.11.2)
Requirement already satisfied: matplotlib!=3.0.0,>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from scikit-image->efficientnet==1.0.0->segmentation_models) (3.2.2)
Requirement already satisfied: scipy>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from scikit-image->efficientnet==1.0.0->segmentation_models) (1.4.1)
Requirement already satisfied: pillow!=7.1.0,!=7.1.1,>=4.3.0 in /usr/local/lib/python3.7/dist-packages (from scikit-image->efficientnet==1.0.0->segmentation_models) (7.1.2)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib!=3.0.0,>=2.0.0->scikit-image->efficientnet==1.0.0->segmentation_models) (1.4.2)
Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib!=3.0.0,>=2.0.0->scikit-image->efficientnet==1.0.0->segmentation_models) (2.8.2)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib!=3.0.0,>=2.0.0->scikit-image->efficientnet==1.0.0->segmentation_models) (0.11.0)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib!=3.0.0,>=2.0.0->scikit-image->efficientnet==1.0.0->segmentation_models) (3.0.8)
Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from kiwisolver>=1.0.1->matplotlib!=3.0.0,>=2.0.0->scikit-image->efficientnet==1.0.0->segmentation_models) (4.2.0)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.1->matplotlib!=3.0.0,>=2.0.0->scikit-image->efficientnet==1.0.0->segmentation_models) (1.15.0)
Installing collected packages: keras-applications, image-classifiers, efficientnet, segmentation-models
Successfully installed efficientnet-1.0.0 image-classifiers-1.0.0 keras-applications-1.0.8 segmentation-models-1.0.1
Segmentation Models: using `keras` framework.
BACKBONE = 'resnet34'
preprocess_input_resnet = get_preprocessing(BACKBONE)
# Apply Pre-processing
Train_X_unet_2 = preprocess_input_resnet(np.copy(Train_X_unet))
Eval_X_unet_2 = preprocess_input_resnet(np.copy(Eval_X_unet))
# Instantiates the Unet architecture with resnet34 as backbone
model_unet_2 = Unet(BACKBONE, input_shape=(128, 128, 3), encoder_weights='imagenet')
# Model summary
model_unet_2.summary()
# Define optimizer and compile the model
model_unet_2.compile(
optimizer = tf.keras.optimizers.Adam(learning_rate=0.0001, beta_1=0.9, beta_2=0.999, epsilon=1e-07),
loss = loss,
metrics = [dice_coefficient])
Downloading data from https://github.com/qubvel/classification_models/releases/download/0.0.1/resnet34_imagenet_1000_no_top.h5
85524480/85521592 [==============================] - 11s 0us/step
85532672/85521592 [==============================] - 11s 0us/step
Model: "model_4"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
data (InputLayer) [(None, 128, 128, 3 0 []
)]
bn_data (BatchNormalization) (None, 128, 128, 3) 9 ['data[0][0]']
zero_padding2d (ZeroPadding2D) (None, 134, 134, 3) 0 ['bn_data[0][0]']
conv0 (Conv2D) (None, 64, 64, 64) 9408 ['zero_padding2d[0][0]']
bn0 (BatchNormalization) (None, 64, 64, 64) 256 ['conv0[0][0]']
relu0 (Activation) (None, 64, 64, 64) 0 ['bn0[0][0]']
zero_padding2d_1 (ZeroPadding2 (None, 66, 66, 64) 0 ['relu0[0][0]']
D)
pooling0 (MaxPooling2D) (None, 32, 32, 64) 0 ['zero_padding2d_1[0][0]']
stage1_unit1_bn1 (BatchNormali (None, 32, 32, 64) 256 ['pooling0[0][0]']
zation)
stage1_unit1_relu1 (Activation (None, 32, 32, 64) 0 ['stage1_unit1_bn1[0][0]']
)
zero_padding2d_2 (ZeroPadding2 (None, 34, 34, 64) 0 ['stage1_unit1_relu1[0][0]']
D)
stage1_unit1_conv1 (Conv2D) (None, 32, 32, 64) 36864 ['zero_padding2d_2[0][0]']
stage1_unit1_bn2 (BatchNormali (None, 32, 32, 64) 256 ['stage1_unit1_conv1[0][0]']
zation)
stage1_unit1_relu2 (Activation (None, 32, 32, 64) 0 ['stage1_unit1_bn2[0][0]']
)
zero_padding2d_3 (ZeroPadding2 (None, 34, 34, 64) 0 ['stage1_unit1_relu2[0][0]']
D)
stage1_unit1_conv2 (Conv2D) (None, 32, 32, 64) 36864 ['zero_padding2d_3[0][0]']
stage1_unit1_sc (Conv2D) (None, 32, 32, 64) 4096 ['stage1_unit1_relu1[0][0]']
add (Add) (None, 32, 32, 64) 0 ['stage1_unit1_conv2[0][0]',
'stage1_unit1_sc[0][0]']
stage1_unit2_bn1 (BatchNormali (None, 32, 32, 64) 256 ['add[0][0]']
zation)
stage1_unit2_relu1 (Activation (None, 32, 32, 64) 0 ['stage1_unit2_bn1[0][0]']
)
zero_padding2d_4 (ZeroPadding2 (None, 34, 34, 64) 0 ['stage1_unit2_relu1[0][0]']
D)
stage1_unit2_conv1 (Conv2D) (None, 32, 32, 64) 36864 ['zero_padding2d_4[0][0]']
stage1_unit2_bn2 (BatchNormali (None, 32, 32, 64) 256 ['stage1_unit2_conv1[0][0]']
zation)
stage1_unit2_relu2 (Activation (None, 32, 32, 64) 0 ['stage1_unit2_bn2[0][0]']
)
zero_padding2d_5 (ZeroPadding2 (None, 34, 34, 64) 0 ['stage1_unit2_relu2[0][0]']
D)
stage1_unit2_conv2 (Conv2D) (None, 32, 32, 64) 36864 ['zero_padding2d_5[0][0]']
add_1 (Add) (None, 32, 32, 64) 0 ['stage1_unit2_conv2[0][0]',
'add[0][0]']
stage1_unit3_bn1 (BatchNormali (None, 32, 32, 64) 256 ['add_1[0][0]']
zation)
stage1_unit3_relu1 (Activation (None, 32, 32, 64) 0 ['stage1_unit3_bn1[0][0]']
)
zero_padding2d_6 (ZeroPadding2 (None, 34, 34, 64) 0 ['stage1_unit3_relu1[0][0]']
D)
stage1_unit3_conv1 (Conv2D) (None, 32, 32, 64) 36864 ['zero_padding2d_6[0][0]']
stage1_unit3_bn2 (BatchNormali (None, 32, 32, 64) 256 ['stage1_unit3_conv1[0][0]']
zation)
stage1_unit3_relu2 (Activation (None, 32, 32, 64) 0 ['stage1_unit3_bn2[0][0]']
)
zero_padding2d_7 (ZeroPadding2 (None, 34, 34, 64) 0 ['stage1_unit3_relu2[0][0]']
D)
stage1_unit3_conv2 (Conv2D) (None, 32, 32, 64) 36864 ['zero_padding2d_7[0][0]']
add_2 (Add) (None, 32, 32, 64) 0 ['stage1_unit3_conv2[0][0]',
'add_1[0][0]']
stage2_unit1_bn1 (BatchNormali (None, 32, 32, 64) 256 ['add_2[0][0]']
zation)
stage2_unit1_relu1 (Activation (None, 32, 32, 64) 0 ['stage2_unit1_bn1[0][0]']
)
zero_padding2d_8 (ZeroPadding2 (None, 34, 34, 64) 0 ['stage2_unit1_relu1[0][0]']
D)
stage2_unit1_conv1 (Conv2D) (None, 16, 16, 128) 73728 ['zero_padding2d_8[0][0]']
stage2_unit1_bn2 (BatchNormali (None, 16, 16, 128) 512 ['stage2_unit1_conv1[0][0]']
zation)
stage2_unit1_relu2 (Activation (None, 16, 16, 128) 0 ['stage2_unit1_bn2[0][0]']
)
zero_padding2d_9 (ZeroPadding2 (None, 18, 18, 128) 0 ['stage2_unit1_relu2[0][0]']
D)
stage2_unit1_conv2 (Conv2D) (None, 16, 16, 128) 147456 ['zero_padding2d_9[0][0]']
stage2_unit1_sc (Conv2D) (None, 16, 16, 128) 8192 ['stage2_unit1_relu1[0][0]']
add_3 (Add) (None, 16, 16, 128) 0 ['stage2_unit1_conv2[0][0]',
'stage2_unit1_sc[0][0]']
stage2_unit2_bn1 (BatchNormali (None, 16, 16, 128) 512 ['add_3[0][0]']
zation)
stage2_unit2_relu1 (Activation (None, 16, 16, 128) 0 ['stage2_unit2_bn1[0][0]']
)
zero_padding2d_10 (ZeroPadding (None, 18, 18, 128) 0 ['stage2_unit2_relu1[0][0]']
2D)
stage2_unit2_conv1 (Conv2D) (None, 16, 16, 128) 147456 ['zero_padding2d_10[0][0]']
stage2_unit2_bn2 (BatchNormali (None, 16, 16, 128) 512 ['stage2_unit2_conv1[0][0]']
zation)
stage2_unit2_relu2 (Activation (None, 16, 16, 128) 0 ['stage2_unit2_bn2[0][0]']
)
zero_padding2d_11 (ZeroPadding (None, 18, 18, 128) 0 ['stage2_unit2_relu2[0][0]']
2D)
stage2_unit2_conv2 (Conv2D) (None, 16, 16, 128) 147456 ['zero_padding2d_11[0][0]']
add_4 (Add) (None, 16, 16, 128) 0 ['stage2_unit2_conv2[0][0]',
'add_3[0][0]']
stage2_unit3_bn1 (BatchNormali (None, 16, 16, 128) 512 ['add_4[0][0]']
zation)
stage2_unit3_relu1 (Activation (None, 16, 16, 128) 0 ['stage2_unit3_bn1[0][0]']
)
zero_padding2d_12 (ZeroPadding (None, 18, 18, 128) 0 ['stage2_unit3_relu1[0][0]']
2D)
stage2_unit3_conv1 (Conv2D) (None, 16, 16, 128) 147456 ['zero_padding2d_12[0][0]']
stage2_unit3_bn2 (BatchNormali (None, 16, 16, 128) 512 ['stage2_unit3_conv1[0][0]']
zation)
stage2_unit3_relu2 (Activation (None, 16, 16, 128) 0 ['stage2_unit3_bn2[0][0]']
)
zero_padding2d_13 (ZeroPadding (None, 18, 18, 128) 0 ['stage2_unit3_relu2[0][0]']
2D)
stage2_unit3_conv2 (Conv2D) (None, 16, 16, 128) 147456 ['zero_padding2d_13[0][0]']
add_5 (Add) (None, 16, 16, 128) 0 ['stage2_unit3_conv2[0][0]',
'add_4[0][0]']
stage2_unit4_bn1 (BatchNormali (None, 16, 16, 128) 512 ['add_5[0][0]']
zation)
stage2_unit4_relu1 (Activation (None, 16, 16, 128) 0 ['stage2_unit4_bn1[0][0]']
)
zero_padding2d_14 (ZeroPadding (None, 18, 18, 128) 0 ['stage2_unit4_relu1[0][0]']
2D)
stage2_unit4_conv1 (Conv2D) (None, 16, 16, 128) 147456 ['zero_padding2d_14[0][0]']
stage2_unit4_bn2 (BatchNormali (None, 16, 16, 128) 512 ['stage2_unit4_conv1[0][0]']
zation)
stage2_unit4_relu2 (Activation (None, 16, 16, 128) 0 ['stage2_unit4_bn2[0][0]']
)
zero_padding2d_15 (ZeroPadding (None, 18, 18, 128) 0 ['stage2_unit4_relu2[0][0]']
2D)
stage2_unit4_conv2 (Conv2D) (None, 16, 16, 128) 147456 ['zero_padding2d_15[0][0]']
add_6 (Add) (None, 16, 16, 128) 0 ['stage2_unit4_conv2[0][0]',
'add_5[0][0]']
stage3_unit1_bn1 (BatchNormali (None, 16, 16, 128) 512 ['add_6[0][0]']
zation)
stage3_unit1_relu1 (Activation (None, 16, 16, 128) 0 ['stage3_unit1_bn1[0][0]']
)
zero_padding2d_16 (ZeroPadding (None, 18, 18, 128) 0 ['stage3_unit1_relu1[0][0]']
2D)
stage3_unit1_conv1 (Conv2D) (None, 8, 8, 256) 294912 ['zero_padding2d_16[0][0]']
stage3_unit1_bn2 (BatchNormali (None, 8, 8, 256) 1024 ['stage3_unit1_conv1[0][0]']
zation)
stage3_unit1_relu2 (Activation (None, 8, 8, 256) 0 ['stage3_unit1_bn2[0][0]']
)
zero_padding2d_17 (ZeroPadding (None, 10, 10, 256) 0 ['stage3_unit1_relu2[0][0]']
2D)
stage3_unit1_conv2 (Conv2D) (None, 8, 8, 256) 589824 ['zero_padding2d_17[0][0]']
stage3_unit1_sc (Conv2D) (None, 8, 8, 256) 32768 ['stage3_unit1_relu1[0][0]']
add_7 (Add) (None, 8, 8, 256) 0 ['stage3_unit1_conv2[0][0]',
'stage3_unit1_sc[0][0]']
stage3_unit2_bn1 (BatchNormali (None, 8, 8, 256) 1024 ['add_7[0][0]']
zation)
stage3_unit2_relu1 (Activation (None, 8, 8, 256) 0 ['stage3_unit2_bn1[0][0]']
)
zero_padding2d_18 (ZeroPadding (None, 10, 10, 256) 0 ['stage3_unit2_relu1[0][0]']
2D)
stage3_unit2_conv1 (Conv2D) (None, 8, 8, 256) 589824 ['zero_padding2d_18[0][0]']
stage3_unit2_bn2 (BatchNormali (None, 8, 8, 256) 1024 ['stage3_unit2_conv1[0][0]']
zation)
stage3_unit2_relu2 (Activation (None, 8, 8, 256) 0 ['stage3_unit2_bn2[0][0]']
)
zero_padding2d_19 (ZeroPadding (None, 10, 10, 256) 0 ['stage3_unit2_relu2[0][0]']
2D)
stage3_unit2_conv2 (Conv2D) (None, 8, 8, 256) 589824 ['zero_padding2d_19[0][0]']
add_8 (Add) (None, 8, 8, 256) 0 ['stage3_unit2_conv2[0][0]',
'add_7[0][0]']
stage3_unit3_bn1 (BatchNormali (None, 8, 8, 256) 1024 ['add_8[0][0]']
zation)
stage3_unit3_relu1 (Activation (None, 8, 8, 256) 0 ['stage3_unit3_bn1[0][0]']
)
zero_padding2d_20 (ZeroPadding (None, 10, 10, 256) 0 ['stage3_unit3_relu1[0][0]']
2D)
stage3_unit3_conv1 (Conv2D) (None, 8, 8, 256) 589824 ['zero_padding2d_20[0][0]']
stage3_unit3_bn2 (BatchNormali (None, 8, 8, 256) 1024 ['stage3_unit3_conv1[0][0]']
zation)
stage3_unit3_relu2 (Activation (None, 8, 8, 256) 0 ['stage3_unit3_bn2[0][0]']
)
zero_padding2d_21 (ZeroPadding (None, 10, 10, 256) 0 ['stage3_unit3_relu2[0][0]']
2D)
stage3_unit3_conv2 (Conv2D) (None, 8, 8, 256) 589824 ['zero_padding2d_21[0][0]']
add_9 (Add) (None, 8, 8, 256) 0 ['stage3_unit3_conv2[0][0]',
'add_8[0][0]']
stage3_unit4_bn1 (BatchNormali (None, 8, 8, 256) 1024 ['add_9[0][0]']
zation)
stage3_unit4_relu1 (Activation (None, 8, 8, 256) 0 ['stage3_unit4_bn1[0][0]']
)
zero_padding2d_22 (ZeroPadding (None, 10, 10, 256) 0 ['stage3_unit4_relu1[0][0]']
2D)
stage3_unit4_conv1 (Conv2D) (None, 8, 8, 256) 589824 ['zero_padding2d_22[0][0]']
stage3_unit4_bn2 (BatchNormali (None, 8, 8, 256) 1024 ['stage3_unit4_conv1[0][0]']
zation)
stage3_unit4_relu2 (Activation (None, 8, 8, 256) 0 ['stage3_unit4_bn2[0][0]']
)
zero_padding2d_23 (ZeroPadding (None, 10, 10, 256) 0 ['stage3_unit4_relu2[0][0]']
2D)
stage3_unit4_conv2 (Conv2D) (None, 8, 8, 256) 589824 ['zero_padding2d_23[0][0]']
add_10 (Add) (None, 8, 8, 256) 0 ['stage3_unit4_conv2[0][0]',
'add_9[0][0]']
stage3_unit5_bn1 (BatchNormali (None, 8, 8, 256) 1024 ['add_10[0][0]']
zation)
stage3_unit5_relu1 (Activation (None, 8, 8, 256) 0 ['stage3_unit5_bn1[0][0]']
)
zero_padding2d_24 (ZeroPadding (None, 10, 10, 256) 0 ['stage3_unit5_relu1[0][0]']
2D)
stage3_unit5_conv1 (Conv2D) (None, 8, 8, 256) 589824 ['zero_padding2d_24[0][0]']
stage3_unit5_bn2 (BatchNormali (None, 8, 8, 256) 1024 ['stage3_unit5_conv1[0][0]']
zation)
stage3_unit5_relu2 (Activation (None, 8, 8, 256) 0 ['stage3_unit5_bn2[0][0]']
)
zero_padding2d_25 (ZeroPadding (None, 10, 10, 256) 0 ['stage3_unit5_relu2[0][0]']
2D)
stage3_unit5_conv2 (Conv2D) (None, 8, 8, 256) 589824 ['zero_padding2d_25[0][0]']
add_11 (Add) (None, 8, 8, 256) 0 ['stage3_unit5_conv2[0][0]',
'add_10[0][0]']
stage3_unit6_bn1 (BatchNormali (None, 8, 8, 256) 1024 ['add_11[0][0]']
zation)
stage3_unit6_relu1 (Activation (None, 8, 8, 256) 0 ['stage3_unit6_bn1[0][0]']
)
zero_padding2d_26 (ZeroPadding (None, 10, 10, 256) 0 ['stage3_unit6_relu1[0][0]']
2D)
stage3_unit6_conv1 (Conv2D) (None, 8, 8, 256) 589824 ['zero_padding2d_26[0][0]']
stage3_unit6_bn2 (BatchNormali (None, 8, 8, 256) 1024 ['stage3_unit6_conv1[0][0]']
zation)
stage3_unit6_relu2 (Activation (None, 8, 8, 256) 0 ['stage3_unit6_bn2[0][0]']
)
zero_padding2d_27 (ZeroPadding (None, 10, 10, 256) 0 ['stage3_unit6_relu2[0][0]']
2D)
stage3_unit6_conv2 (Conv2D) (None, 8, 8, 256) 589824 ['zero_padding2d_27[0][0]']
add_12 (Add) (None, 8, 8, 256) 0 ['stage3_unit6_conv2[0][0]',
'add_11[0][0]']
stage4_unit1_bn1 (BatchNormali (None, 8, 8, 256) 1024 ['add_12[0][0]']
zation)
stage4_unit1_relu1 (Activation (None, 8, 8, 256) 0 ['stage4_unit1_bn1[0][0]']
)
zero_padding2d_28 (ZeroPadding (None, 10, 10, 256) 0 ['stage4_unit1_relu1[0][0]']
2D)
stage4_unit1_conv1 (Conv2D) (None, 4, 4, 512) 1179648 ['zero_padding2d_28[0][0]']
stage4_unit1_bn2 (BatchNormali (None, 4, 4, 512) 2048 ['stage4_unit1_conv1[0][0]']
zation)
stage4_unit1_relu2 (Activation (None, 4, 4, 512) 0 ['stage4_unit1_bn2[0][0]']
)
zero_padding2d_29 (ZeroPadding (None, 6, 6, 512) 0 ['stage4_unit1_relu2[0][0]']
2D)
stage4_unit1_conv2 (Conv2D) (None, 4, 4, 512) 2359296 ['zero_padding2d_29[0][0]']
stage4_unit1_sc (Conv2D) (None, 4, 4, 512) 131072 ['stage4_unit1_relu1[0][0]']
add_13 (Add) (None, 4, 4, 512) 0 ['stage4_unit1_conv2[0][0]',
'stage4_unit1_sc[0][0]']
stage4_unit2_bn1 (BatchNormali (None, 4, 4, 512) 2048 ['add_13[0][0]']
zation)
stage4_unit2_relu1 (Activation (None, 4, 4, 512) 0 ['stage4_unit2_bn1[0][0]']
)
zero_padding2d_30 (ZeroPadding (None, 6, 6, 512) 0 ['stage4_unit2_relu1[0][0]']
2D)
stage4_unit2_conv1 (Conv2D) (None, 4, 4, 512) 2359296 ['zero_padding2d_30[0][0]']
stage4_unit2_bn2 (BatchNormali (None, 4, 4, 512) 2048 ['stage4_unit2_conv1[0][0]']
zation)
stage4_unit2_relu2 (Activation (None, 4, 4, 512) 0 ['stage4_unit2_bn2[0][0]']
)
zero_padding2d_31 (ZeroPadding (None, 6, 6, 512) 0 ['stage4_unit2_relu2[0][0]']
2D)
stage4_unit2_conv2 (Conv2D) (None, 4, 4, 512) 2359296 ['zero_padding2d_31[0][0]']
add_14 (Add) (None, 4, 4, 512) 0 ['stage4_unit2_conv2[0][0]',
'add_13[0][0]']
stage4_unit3_bn1 (BatchNormali (None, 4, 4, 512) 2048 ['add_14[0][0]']
zation)
stage4_unit3_relu1 (Activation (None, 4, 4, 512) 0 ['stage4_unit3_bn1[0][0]']
)
zero_padding2d_32 (ZeroPadding (None, 6, 6, 512) 0 ['stage4_unit3_relu1[0][0]']
2D)
stage4_unit3_conv1 (Conv2D) (None, 4, 4, 512) 2359296 ['zero_padding2d_32[0][0]']
stage4_unit3_bn2 (BatchNormali (None, 4, 4, 512) 2048 ['stage4_unit3_conv1[0][0]']
zation)
stage4_unit3_relu2 (Activation (None, 4, 4, 512) 0 ['stage4_unit3_bn2[0][0]']
)
zero_padding2d_33 (ZeroPadding (None, 6, 6, 512) 0 ['stage4_unit3_relu2[0][0]']
2D)
stage4_unit3_conv2 (Conv2D) (None, 4, 4, 512) 2359296 ['zero_padding2d_33[0][0]']
add_15 (Add) (None, 4, 4, 512) 0 ['stage4_unit3_conv2[0][0]',
'add_14[0][0]']
bn1 (BatchNormalization) (None, 4, 4, 512) 2048 ['add_15[0][0]']
relu1 (Activation) (None, 4, 4, 512) 0 ['bn1[0][0]']
decoder_stage0_upsampling (UpS (None, 8, 8, 512) 0 ['relu1[0][0]']
ampling2D)
decoder_stage0_concat (Concate (None, 8, 8, 768) 0 ['decoder_stage0_upsampling[0][0]
nate) ',
'stage4_unit1_relu1[0][0]']
decoder_stage0a_conv (Conv2D) (None, 8, 8, 256) 1769472 ['decoder_stage0_concat[0][0]']
decoder_stage0a_bn (BatchNorma (None, 8, 8, 256) 1024 ['decoder_stage0a_conv[0][0]']
lization)
decoder_stage0a_relu (Activati (None, 8, 8, 256) 0 ['decoder_stage0a_bn[0][0]']
on)
decoder_stage0b_conv (Conv2D) (None, 8, 8, 256) 589824 ['decoder_stage0a_relu[0][0]']
decoder_stage0b_bn (BatchNorma (None, 8, 8, 256) 1024 ['decoder_stage0b_conv[0][0]']
lization)
decoder_stage0b_relu (Activati (None, 8, 8, 256) 0 ['decoder_stage0b_bn[0][0]']
on)
decoder_stage1_upsampling (UpS (None, 16, 16, 256) 0 ['decoder_stage0b_relu[0][0]']
ampling2D)
decoder_stage1_concat (Concate (None, 16, 16, 384) 0 ['decoder_stage1_upsampling[0][0]
nate) ',
'stage3_unit1_relu1[0][0]']
decoder_stage1a_conv (Conv2D) (None, 16, 16, 128) 442368 ['decoder_stage1_concat[0][0]']
decoder_stage1a_bn (BatchNorma (None, 16, 16, 128) 512 ['decoder_stage1a_conv[0][0]']
lization)
decoder_stage1a_relu (Activati (None, 16, 16, 128) 0 ['decoder_stage1a_bn[0][0]']
on)
decoder_stage1b_conv (Conv2D) (None, 16, 16, 128) 147456 ['decoder_stage1a_relu[0][0]']
decoder_stage1b_bn (BatchNorma (None, 16, 16, 128) 512 ['decoder_stage1b_conv[0][0]']
lization)
decoder_stage1b_relu (Activati (None, 16, 16, 128) 0 ['decoder_stage1b_bn[0][0]']
on)
decoder_stage2_upsampling (UpS (None, 32, 32, 128) 0 ['decoder_stage1b_relu[0][0]']
ampling2D)
decoder_stage2_concat (Concate (None, 32, 32, 192) 0 ['decoder_stage2_upsampling[0][0]
nate) ',
'stage2_unit1_relu1[0][0]']
decoder_stage2a_conv (Conv2D) (None, 32, 32, 64) 110592 ['decoder_stage2_concat[0][0]']
decoder_stage2a_bn (BatchNorma (None, 32, 32, 64) 256 ['decoder_stage2a_conv[0][0]']
lization)
decoder_stage2a_relu (Activati (None, 32, 32, 64) 0 ['decoder_stage2a_bn[0][0]']
on)
decoder_stage2b_conv (Conv2D) (None, 32, 32, 64) 36864 ['decoder_stage2a_relu[0][0]']
decoder_stage2b_bn (BatchNorma (None, 32, 32, 64) 256 ['decoder_stage2b_conv[0][0]']
lization)
decoder_stage2b_relu (Activati (None, 32, 32, 64) 0 ['decoder_stage2b_bn[0][0]']
on)
decoder_stage3_upsampling (UpS (None, 64, 64, 64) 0 ['decoder_stage2b_relu[0][0]']
ampling2D)
decoder_stage3_concat (Concate (None, 64, 64, 128) 0 ['decoder_stage3_upsampling[0][0]
nate) ',
'relu0[0][0]']
decoder_stage3a_conv (Conv2D) (None, 64, 64, 32) 36864 ['decoder_stage3_concat[0][0]']
decoder_stage3a_bn (BatchNorma (None, 64, 64, 32) 128 ['decoder_stage3a_conv[0][0]']
lization)
decoder_stage3a_relu (Activati (None, 64, 64, 32) 0 ['decoder_stage3a_bn[0][0]']
on)
decoder_stage3b_conv (Conv2D) (None, 64, 64, 32) 9216 ['decoder_stage3a_relu[0][0]']
decoder_stage3b_bn (BatchNorma (None, 64, 64, 32) 128 ['decoder_stage3b_conv[0][0]']
lization)
decoder_stage3b_relu (Activati (None, 64, 64, 32) 0 ['decoder_stage3b_bn[0][0]']
on)
decoder_stage4_upsampling (UpS (None, 128, 128, 32 0 ['decoder_stage3b_relu[0][0]']
ampling2D) )
decoder_stage4a_conv (Conv2D) (None, 128, 128, 16 4608 ['decoder_stage4_upsampling[0][0]
) ']
decoder_stage4a_bn (BatchNorma (None, 128, 128, 16 64 ['decoder_stage4a_conv[0][0]']
lization) )
decoder_stage4a_relu (Activati (None, 128, 128, 16 0 ['decoder_stage4a_bn[0][0]']
on) )
decoder_stage4b_conv (Conv2D) (None, 128, 128, 16 2304 ['decoder_stage4a_relu[0][0]']
)
decoder_stage4b_bn (BatchNorma (None, 128, 128, 16 64 ['decoder_stage4b_conv[0][0]']
lization) )
decoder_stage4b_relu (Activati (None, 128, 128, 16 0 ['decoder_stage4b_bn[0][0]']
on) )
final_conv (Conv2D) (None, 128, 128, 1) 145 ['decoder_stage4b_relu[0][0]']
sigmoid (Activation) (None, 128, 128, 1) 0 ['final_conv[0][0]']
==================================================================================================
Total params: 24,456,154
Trainable params: 24,438,804
Non-trainable params: 17,350
__________________________________________________________________________________________________
# Plot the model
tf.keras.utils.plot_model(model_unet_2, show_shapes=True)
# Fit the model
history_unet_2 = model_unet_2.fit(Train_X_unet_2, Train_mask_unet, epochs=20, batch_size=32, validation_split=0.35, callbacks=[early_stop, reduce_lr])
Epoch 1/20 69/69 [==============================] - 22s 221ms/step - loss: 1.5066 - dice_coefficient: 0.3769 - val_loss: 1.6007 - val_dice_coefficient: 0.3517 - lr: 1.0000e-04 Epoch 2/20 69/69 [==============================] - 11s 163ms/step - loss: 1.0388 - dice_coefficient: 0.5048 - val_loss: 1.2645 - val_dice_coefficient: 0.4694 - lr: 1.0000e-04 Epoch 3/20 69/69 [==============================] - 11s 161ms/step - loss: 0.7888 - dice_coefficient: 0.5909 - val_loss: 0.9926 - val_dice_coefficient: 0.5369 - lr: 1.0000e-04 Epoch 4/20 69/69 [==============================] - 11s 162ms/step - loss: 0.6352 - dice_coefficient: 0.6547 - val_loss: 0.8642 - val_dice_coefficient: 0.5896 - lr: 1.0000e-04 Epoch 5/20 69/69 [==============================] - 11s 162ms/step - loss: 0.5338 - dice_coefficient: 0.7014 - val_loss: 0.7557 - val_dice_coefficient: 0.6236 - lr: 1.0000e-04 Epoch 6/20 69/69 [==============================] - 11s 163ms/step - loss: 0.4579 - dice_coefficient: 0.7394 - val_loss: 0.7245 - val_dice_coefficient: 0.6483 - lr: 1.0000e-04 Epoch 7/20 69/69 [==============================] - 11s 164ms/step - loss: 0.3984 - dice_coefficient: 0.7710 - val_loss: 0.7147 - val_dice_coefficient: 0.6421 - lr: 1.0000e-04 Epoch 8/20 69/69 [==============================] - 11s 163ms/step - loss: 0.3514 - dice_coefficient: 0.7962 - val_loss: 0.6657 - val_dice_coefficient: 0.6714 - lr: 1.0000e-04 Epoch 9/20 69/69 [==============================] - 11s 163ms/step - loss: 0.3158 - dice_coefficient: 0.8163 - val_loss: 0.6920 - val_dice_coefficient: 0.6611 - lr: 1.0000e-04 Epoch 10/20 69/69 [==============================] - 11s 163ms/step - loss: 0.2838 - dice_coefficient: 0.8342 - val_loss: 0.6770 - val_dice_coefficient: 0.6804 - lr: 1.0000e-04 Epoch 11/20 69/69 [==============================] - 11s 163ms/step - loss: 0.2571 - dice_coefficient: 0.8494 - val_loss: 0.6913 - val_dice_coefficient: 0.6748 - lr: 1.0000e-04 Epoch 12/20 69/69 [==============================] - 11s 164ms/step - loss: 0.2332 - dice_coefficient: 0.8627 - val_loss: 0.7022 - val_dice_coefficient: 0.6748 - lr: 1.0000e-04 Epoch 13/20 69/69 [==============================] - 11s 163ms/step - loss: 0.2167 - dice_coefficient: 0.8726 - val_loss: 0.6726 - val_dice_coefficient: 0.6927 - lr: 1.0000e-04 Epoch 14/20 69/69 [==============================] - 11s 164ms/step - loss: 0.2038 - dice_coefficient: 0.8804 - val_loss: 0.6892 - val_dice_coefficient: 0.6842 - lr: 1.0000e-04 Epoch 15/20 69/69 [==============================] - 11s 164ms/step - loss: 0.1891 - dice_coefficient: 0.8888 - val_loss: 0.6857 - val_dice_coefficient: 0.6907 - lr: 1.0000e-04 Epoch 16/20 69/69 [==============================] - 11s 164ms/step - loss: 0.1817 - dice_coefficient: 0.8934 - val_loss: 0.6876 - val_dice_coefficient: 0.6952 - lr: 1.0000e-04 Epoch 17/20 69/69 [==============================] - 11s 164ms/step - loss: 0.1722 - dice_coefficient: 0.8990 - val_loss: 0.6775 - val_dice_coefficient: 0.7017 - lr: 1.0000e-04 Epoch 18/20 69/69 [==============================] - 11s 164ms/step - loss: 0.1600 - dice_coefficient: 0.9058 - val_loss: 0.6958 - val_dice_coefficient: 0.6983 - lr: 1.0000e-04 Epoch 19/20 69/69 [==============================] - 11s 164ms/step - loss: 0.1455 - dice_coefficient: 0.9140 - val_loss: 0.7010 - val_dice_coefficient: 0.6927 - lr: 1.0000e-04 Epoch 20/20 69/69 [==============================] - 11s 164ms/step - loss: 0.1430 - dice_coefficient: 0.9158 - val_loss: 0.7170 - val_dice_coefficient: 0.6920 - lr: 1.0000e-04
# Plot accuracy and loss
plot_accuracy_loss(history=history_unet_2, model_type='UNET')
# Model Evaluation
eval_acc_unet_2, eval_loss_unet_2, train_acc_unet_2, train_loss_unet_2 = evaluate_model(model_unet_2, Eval_X_unet_2, Eval_mask_unet, Train_X_unet_2, Train_mask_unet, 'UNET')
1/1 [==============================] - 1s 570ms/step - loss: 0.8286 - dice_coefficient: 0.6461 106/106 [==============================] - 5s 48ms/step - loss: 0.3443 - dice_coefficient: 0.8346 Dice Coefficient on Eval set = 0.6461 & Loss on Eval set = 0.8286
# Append the scores & loss
unet_model_scores['Model'].append('Resnet34')
unet_model_scores['Train Score'].append(train_acc_unet_2)
unet_model_scores['Eval Score'].append(eval_acc_unet_2)
unet_model_scores['Train Loss'].append(train_loss_unet_2)
unet_model_scores['Eval Loss'].append(eval_loss_unet_2)
del Train_X_unet_2
del Eval_X_unet_2
BACKBONE = 'inceptionv3'
preprocess_input_incep = get_preprocessing(BACKBONE)
# Apply Pre-processing
Train_X_unet_3 = preprocess_input_incep(np.copy(Train_X_unet))
Eval_X_unet_3 = preprocess_input_incep(np.copy(Eval_X_unet))
# Instantiates the Unet architecture with inceptionv3 as backbone
model_unet_3 = Unet(BACKBONE, input_shape=(128, 128, 3), encoder_weights='imagenet')
# Model summary
model_unet_3.summary()
# Define optimizer and compile the model
model_unet_3.compile(
optimizer = tf.keras.optimizers.Adam(learning_rate=0.0001, beta_1=0.9, beta_2=0.999, epsilon=1e-07),
loss = loss,
metrics = [dice_coefficient])
Model: "model_5"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_4 (InputLayer) [(None, 128, 128, 3 0 []
)]
conv2d_99 (Conv2D) (None, 64, 64, 32) 864 ['input_4[0][0]']
batch_normalization_95 (BatchN (None, 64, 64, 32) 96 ['conv2d_99[0][0]']
ormalization)
activation_94 (Activation) (None, 64, 64, 32) 0 ['batch_normalization_95[0][0]']
conv2d_100 (Conv2D) (None, 64, 64, 32) 9216 ['activation_94[0][0]']
batch_normalization_96 (BatchN (None, 64, 64, 32) 96 ['conv2d_100[0][0]']
ormalization)
activation_95 (Activation) (None, 64, 64, 32) 0 ['batch_normalization_96[0][0]']
conv2d_101 (Conv2D) (None, 64, 64, 64) 18432 ['activation_95[0][0]']
batch_normalization_97 (BatchN (None, 64, 64, 64) 192 ['conv2d_101[0][0]']
ormalization)
activation_96 (Activation) (None, 64, 64, 64) 0 ['batch_normalization_97[0][0]']
max_pooling2d_8 (MaxPooling2D) (None, 32, 32, 64) 0 ['activation_96[0][0]']
conv2d_102 (Conv2D) (None, 32, 32, 80) 5120 ['max_pooling2d_8[0][0]']
batch_normalization_98 (BatchN (None, 32, 32, 80) 240 ['conv2d_102[0][0]']
ormalization)
activation_97 (Activation) (None, 32, 32, 80) 0 ['batch_normalization_98[0][0]']
conv2d_103 (Conv2D) (None, 32, 32, 192) 138240 ['activation_97[0][0]']
batch_normalization_99 (BatchN (None, 32, 32, 192) 576 ['conv2d_103[0][0]']
ormalization)
activation_98 (Activation) (None, 32, 32, 192) 0 ['batch_normalization_99[0][0]']
max_pooling2d_9 (MaxPooling2D) (None, 16, 16, 192) 0 ['activation_98[0][0]']
conv2d_107 (Conv2D) (None, 16, 16, 64) 12288 ['max_pooling2d_9[0][0]']
batch_normalization_103 (Batch (None, 16, 16, 64) 192 ['conv2d_107[0][0]']
Normalization)
activation_102 (Activation) (None, 16, 16, 64) 0 ['batch_normalization_103[0][0]']
conv2d_105 (Conv2D) (None, 16, 16, 48) 9216 ['max_pooling2d_9[0][0]']
conv2d_108 (Conv2D) (None, 16, 16, 96) 55296 ['activation_102[0][0]']
batch_normalization_101 (Batch (None, 16, 16, 48) 144 ['conv2d_105[0][0]']
Normalization)
batch_normalization_104 (Batch (None, 16, 16, 96) 288 ['conv2d_108[0][0]']
Normalization)
activation_100 (Activation) (None, 16, 16, 48) 0 ['batch_normalization_101[0][0]']
activation_103 (Activation) (None, 16, 16, 96) 0 ['batch_normalization_104[0][0]']
average_pooling2d_9 (AveragePo (None, 16, 16, 192) 0 ['max_pooling2d_9[0][0]']
oling2D)
conv2d_104 (Conv2D) (None, 16, 16, 64) 12288 ['max_pooling2d_9[0][0]']
conv2d_106 (Conv2D) (None, 16, 16, 64) 76800 ['activation_100[0][0]']
conv2d_109 (Conv2D) (None, 16, 16, 96) 82944 ['activation_103[0][0]']
conv2d_110 (Conv2D) (None, 16, 16, 32) 6144 ['average_pooling2d_9[0][0]']
batch_normalization_100 (Batch (None, 16, 16, 64) 192 ['conv2d_104[0][0]']
Normalization)
batch_normalization_102 (Batch (None, 16, 16, 64) 192 ['conv2d_106[0][0]']
Normalization)
batch_normalization_105 (Batch (None, 16, 16, 96) 288 ['conv2d_109[0][0]']
Normalization)
batch_normalization_106 (Batch (None, 16, 16, 32) 96 ['conv2d_110[0][0]']
Normalization)
activation_99 (Activation) (None, 16, 16, 64) 0 ['batch_normalization_100[0][0]']
activation_101 (Activation) (None, 16, 16, 64) 0 ['batch_normalization_102[0][0]']
activation_104 (Activation) (None, 16, 16, 96) 0 ['batch_normalization_105[0][0]']
activation_105 (Activation) (None, 16, 16, 32) 0 ['batch_normalization_106[0][0]']
mixed0 (Concatenate) (None, 16, 16, 256) 0 ['activation_99[0][0]',
'activation_101[0][0]',
'activation_104[0][0]',
'activation_105[0][0]']
conv2d_114 (Conv2D) (None, 16, 16, 64) 16384 ['mixed0[0][0]']
batch_normalization_110 (Batch (None, 16, 16, 64) 192 ['conv2d_114[0][0]']
Normalization)
activation_109 (Activation) (None, 16, 16, 64) 0 ['batch_normalization_110[0][0]']
conv2d_112 (Conv2D) (None, 16, 16, 48) 12288 ['mixed0[0][0]']
conv2d_115 (Conv2D) (None, 16, 16, 96) 55296 ['activation_109[0][0]']
batch_normalization_108 (Batch (None, 16, 16, 48) 144 ['conv2d_112[0][0]']
Normalization)
batch_normalization_111 (Batch (None, 16, 16, 96) 288 ['conv2d_115[0][0]']
Normalization)
activation_107 (Activation) (None, 16, 16, 48) 0 ['batch_normalization_108[0][0]']
activation_110 (Activation) (None, 16, 16, 96) 0 ['batch_normalization_111[0][0]']
average_pooling2d_10 (AverageP (None, 16, 16, 256) 0 ['mixed0[0][0]']
ooling2D)
conv2d_111 (Conv2D) (None, 16, 16, 64) 16384 ['mixed0[0][0]']
conv2d_113 (Conv2D) (None, 16, 16, 64) 76800 ['activation_107[0][0]']
conv2d_116 (Conv2D) (None, 16, 16, 96) 82944 ['activation_110[0][0]']
conv2d_117 (Conv2D) (None, 16, 16, 64) 16384 ['average_pooling2d_10[0][0]']
batch_normalization_107 (Batch (None, 16, 16, 64) 192 ['conv2d_111[0][0]']
Normalization)
batch_normalization_109 (Batch (None, 16, 16, 64) 192 ['conv2d_113[0][0]']
Normalization)
batch_normalization_112 (Batch (None, 16, 16, 96) 288 ['conv2d_116[0][0]']
Normalization)
batch_normalization_113 (Batch (None, 16, 16, 64) 192 ['conv2d_117[0][0]']
Normalization)
activation_106 (Activation) (None, 16, 16, 64) 0 ['batch_normalization_107[0][0]']
activation_108 (Activation) (None, 16, 16, 64) 0 ['batch_normalization_109[0][0]']
activation_111 (Activation) (None, 16, 16, 96) 0 ['batch_normalization_112[0][0]']
activation_112 (Activation) (None, 16, 16, 64) 0 ['batch_normalization_113[0][0]']
mixed1 (Concatenate) (None, 16, 16, 288) 0 ['activation_106[0][0]',
'activation_108[0][0]',
'activation_111[0][0]',
'activation_112[0][0]']
conv2d_121 (Conv2D) (None, 16, 16, 64) 18432 ['mixed1[0][0]']
batch_normalization_117 (Batch (None, 16, 16, 64) 192 ['conv2d_121[0][0]']
Normalization)
activation_116 (Activation) (None, 16, 16, 64) 0 ['batch_normalization_117[0][0]']
conv2d_119 (Conv2D) (None, 16, 16, 48) 13824 ['mixed1[0][0]']
conv2d_122 (Conv2D) (None, 16, 16, 96) 55296 ['activation_116[0][0]']
batch_normalization_115 (Batch (None, 16, 16, 48) 144 ['conv2d_119[0][0]']
Normalization)
batch_normalization_118 (Batch (None, 16, 16, 96) 288 ['conv2d_122[0][0]']
Normalization)
activation_114 (Activation) (None, 16, 16, 48) 0 ['batch_normalization_115[0][0]']
activation_117 (Activation) (None, 16, 16, 96) 0 ['batch_normalization_118[0][0]']
average_pooling2d_11 (AverageP (None, 16, 16, 288) 0 ['mixed1[0][0]']
ooling2D)
conv2d_118 (Conv2D) (None, 16, 16, 64) 18432 ['mixed1[0][0]']
conv2d_120 (Conv2D) (None, 16, 16, 64) 76800 ['activation_114[0][0]']
conv2d_123 (Conv2D) (None, 16, 16, 96) 82944 ['activation_117[0][0]']
conv2d_124 (Conv2D) (None, 16, 16, 64) 18432 ['average_pooling2d_11[0][0]']
batch_normalization_114 (Batch (None, 16, 16, 64) 192 ['conv2d_118[0][0]']
Normalization)
batch_normalization_116 (Batch (None, 16, 16, 64) 192 ['conv2d_120[0][0]']
Normalization)
batch_normalization_119 (Batch (None, 16, 16, 96) 288 ['conv2d_123[0][0]']
Normalization)
batch_normalization_120 (Batch (None, 16, 16, 64) 192 ['conv2d_124[0][0]']
Normalization)
activation_113 (Activation) (None, 16, 16, 64) 0 ['batch_normalization_114[0][0]']
activation_115 (Activation) (None, 16, 16, 64) 0 ['batch_normalization_116[0][0]']
activation_118 (Activation) (None, 16, 16, 96) 0 ['batch_normalization_119[0][0]']
activation_119 (Activation) (None, 16, 16, 64) 0 ['batch_normalization_120[0][0]']
mixed2 (Concatenate) (None, 16, 16, 288) 0 ['activation_113[0][0]',
'activation_115[0][0]',
'activation_118[0][0]',
'activation_119[0][0]']
conv2d_126 (Conv2D) (None, 16, 16, 64) 18432 ['mixed2[0][0]']
batch_normalization_122 (Batch (None, 16, 16, 64) 192 ['conv2d_126[0][0]']
Normalization)
activation_121 (Activation) (None, 16, 16, 64) 0 ['batch_normalization_122[0][0]']
conv2d_127 (Conv2D) (None, 16, 16, 96) 55296 ['activation_121[0][0]']
batch_normalization_123 (Batch (None, 16, 16, 96) 288 ['conv2d_127[0][0]']
Normalization)
activation_122 (Activation) (None, 16, 16, 96) 0 ['batch_normalization_123[0][0]']
conv2d_125 (Conv2D) (None, 8, 8, 384) 995328 ['mixed2[0][0]']
conv2d_128 (Conv2D) (None, 8, 8, 96) 82944 ['activation_122[0][0]']
batch_normalization_121 (Batch (None, 8, 8, 384) 1152 ['conv2d_125[0][0]']
Normalization)
batch_normalization_124 (Batch (None, 8, 8, 96) 288 ['conv2d_128[0][0]']
Normalization)
activation_120 (Activation) (None, 8, 8, 384) 0 ['batch_normalization_121[0][0]']
activation_123 (Activation) (None, 8, 8, 96) 0 ['batch_normalization_124[0][0]']
max_pooling2d_10 (MaxPooling2D (None, 8, 8, 288) 0 ['mixed2[0][0]']
)
mixed3 (Concatenate) (None, 8, 8, 768) 0 ['activation_120[0][0]',
'activation_123[0][0]',
'max_pooling2d_10[0][0]']
conv2d_133 (Conv2D) (None, 8, 8, 128) 98304 ['mixed3[0][0]']
batch_normalization_129 (Batch (None, 8, 8, 128) 384 ['conv2d_133[0][0]']
Normalization)
activation_128 (Activation) (None, 8, 8, 128) 0 ['batch_normalization_129[0][0]']
conv2d_134 (Conv2D) (None, 8, 8, 128) 114688 ['activation_128[0][0]']
batch_normalization_130 (Batch (None, 8, 8, 128) 384 ['conv2d_134[0][0]']
Normalization)
activation_129 (Activation) (None, 8, 8, 128) 0 ['batch_normalization_130[0][0]']
conv2d_130 (Conv2D) (None, 8, 8, 128) 98304 ['mixed3[0][0]']
conv2d_135 (Conv2D) (None, 8, 8, 128) 114688 ['activation_129[0][0]']
batch_normalization_126 (Batch (None, 8, 8, 128) 384 ['conv2d_130[0][0]']
Normalization)
batch_normalization_131 (Batch (None, 8, 8, 128) 384 ['conv2d_135[0][0]']
Normalization)
activation_125 (Activation) (None, 8, 8, 128) 0 ['batch_normalization_126[0][0]']
activation_130 (Activation) (None, 8, 8, 128) 0 ['batch_normalization_131[0][0]']
conv2d_131 (Conv2D) (None, 8, 8, 128) 114688 ['activation_125[0][0]']
conv2d_136 (Conv2D) (None, 8, 8, 128) 114688 ['activation_130[0][0]']
batch_normalization_127 (Batch (None, 8, 8, 128) 384 ['conv2d_131[0][0]']
Normalization)
batch_normalization_132 (Batch (None, 8, 8, 128) 384 ['conv2d_136[0][0]']
Normalization)
activation_126 (Activation) (None, 8, 8, 128) 0 ['batch_normalization_127[0][0]']
activation_131 (Activation) (None, 8, 8, 128) 0 ['batch_normalization_132[0][0]']
average_pooling2d_12 (AverageP (None, 8, 8, 768) 0 ['mixed3[0][0]']
ooling2D)
conv2d_129 (Conv2D) (None, 8, 8, 192) 147456 ['mixed3[0][0]']
conv2d_132 (Conv2D) (None, 8, 8, 192) 172032 ['activation_126[0][0]']
conv2d_137 (Conv2D) (None, 8, 8, 192) 172032 ['activation_131[0][0]']
conv2d_138 (Conv2D) (None, 8, 8, 192) 147456 ['average_pooling2d_12[0][0]']
batch_normalization_125 (Batch (None, 8, 8, 192) 576 ['conv2d_129[0][0]']
Normalization)
batch_normalization_128 (Batch (None, 8, 8, 192) 576 ['conv2d_132[0][0]']
Normalization)
batch_normalization_133 (Batch (None, 8, 8, 192) 576 ['conv2d_137[0][0]']
Normalization)
batch_normalization_134 (Batch (None, 8, 8, 192) 576 ['conv2d_138[0][0]']
Normalization)
activation_124 (Activation) (None, 8, 8, 192) 0 ['batch_normalization_125[0][0]']
activation_127 (Activation) (None, 8, 8, 192) 0 ['batch_normalization_128[0][0]']
activation_132 (Activation) (None, 8, 8, 192) 0 ['batch_normalization_133[0][0]']
activation_133 (Activation) (None, 8, 8, 192) 0 ['batch_normalization_134[0][0]']
mixed4 (Concatenate) (None, 8, 8, 768) 0 ['activation_124[0][0]',
'activation_127[0][0]',
'activation_132[0][0]',
'activation_133[0][0]']
conv2d_143 (Conv2D) (None, 8, 8, 160) 122880 ['mixed4[0][0]']
batch_normalization_139 (Batch (None, 8, 8, 160) 480 ['conv2d_143[0][0]']
Normalization)
activation_138 (Activation) (None, 8, 8, 160) 0 ['batch_normalization_139[0][0]']
conv2d_144 (Conv2D) (None, 8, 8, 160) 179200 ['activation_138[0][0]']
batch_normalization_140 (Batch (None, 8, 8, 160) 480 ['conv2d_144[0][0]']
Normalization)
activation_139 (Activation) (None, 8, 8, 160) 0 ['batch_normalization_140[0][0]']
conv2d_140 (Conv2D) (None, 8, 8, 160) 122880 ['mixed4[0][0]']
conv2d_145 (Conv2D) (None, 8, 8, 160) 179200 ['activation_139[0][0]']
batch_normalization_136 (Batch (None, 8, 8, 160) 480 ['conv2d_140[0][0]']
Normalization)
batch_normalization_141 (Batch (None, 8, 8, 160) 480 ['conv2d_145[0][0]']
Normalization)
activation_135 (Activation) (None, 8, 8, 160) 0 ['batch_normalization_136[0][0]']
activation_140 (Activation) (None, 8, 8, 160) 0 ['batch_normalization_141[0][0]']
conv2d_141 (Conv2D) (None, 8, 8, 160) 179200 ['activation_135[0][0]']
conv2d_146 (Conv2D) (None, 8, 8, 160) 179200 ['activation_140[0][0]']
batch_normalization_137 (Batch (None, 8, 8, 160) 480 ['conv2d_141[0][0]']
Normalization)
batch_normalization_142 (Batch (None, 8, 8, 160) 480 ['conv2d_146[0][0]']
Normalization)
activation_136 (Activation) (None, 8, 8, 160) 0 ['batch_normalization_137[0][0]']
activation_141 (Activation) (None, 8, 8, 160) 0 ['batch_normalization_142[0][0]']
average_pooling2d_13 (AverageP (None, 8, 8, 768) 0 ['mixed4[0][0]']
ooling2D)
conv2d_139 (Conv2D) (None, 8, 8, 192) 147456 ['mixed4[0][0]']
conv2d_142 (Conv2D) (None, 8, 8, 192) 215040 ['activation_136[0][0]']
conv2d_147 (Conv2D) (None, 8, 8, 192) 215040 ['activation_141[0][0]']
conv2d_148 (Conv2D) (None, 8, 8, 192) 147456 ['average_pooling2d_13[0][0]']
batch_normalization_135 (Batch (None, 8, 8, 192) 576 ['conv2d_139[0][0]']
Normalization)
batch_normalization_138 (Batch (None, 8, 8, 192) 576 ['conv2d_142[0][0]']
Normalization)
batch_normalization_143 (Batch (None, 8, 8, 192) 576 ['conv2d_147[0][0]']
Normalization)
batch_normalization_144 (Batch (None, 8, 8, 192) 576 ['conv2d_148[0][0]']
Normalization)
activation_134 (Activation) (None, 8, 8, 192) 0 ['batch_normalization_135[0][0]']
activation_137 (Activation) (None, 8, 8, 192) 0 ['batch_normalization_138[0][0]']
activation_142 (Activation) (None, 8, 8, 192) 0 ['batch_normalization_143[0][0]']
activation_143 (Activation) (None, 8, 8, 192) 0 ['batch_normalization_144[0][0]']
mixed5 (Concatenate) (None, 8, 8, 768) 0 ['activation_134[0][0]',
'activation_137[0][0]',
'activation_142[0][0]',
'activation_143[0][0]']
conv2d_153 (Conv2D) (None, 8, 8, 160) 122880 ['mixed5[0][0]']
batch_normalization_149 (Batch (None, 8, 8, 160) 480 ['conv2d_153[0][0]']
Normalization)
activation_148 (Activation) (None, 8, 8, 160) 0 ['batch_normalization_149[0][0]']
conv2d_154 (Conv2D) (None, 8, 8, 160) 179200 ['activation_148[0][0]']
batch_normalization_150 (Batch (None, 8, 8, 160) 480 ['conv2d_154[0][0]']
Normalization)
activation_149 (Activation) (None, 8, 8, 160) 0 ['batch_normalization_150[0][0]']
conv2d_150 (Conv2D) (None, 8, 8, 160) 122880 ['mixed5[0][0]']
conv2d_155 (Conv2D) (None, 8, 8, 160) 179200 ['activation_149[0][0]']
batch_normalization_146 (Batch (None, 8, 8, 160) 480 ['conv2d_150[0][0]']
Normalization)
batch_normalization_151 (Batch (None, 8, 8, 160) 480 ['conv2d_155[0][0]']
Normalization)
activation_145 (Activation) (None, 8, 8, 160) 0 ['batch_normalization_146[0][0]']
activation_150 (Activation) (None, 8, 8, 160) 0 ['batch_normalization_151[0][0]']
conv2d_151 (Conv2D) (None, 8, 8, 160) 179200 ['activation_145[0][0]']
conv2d_156 (Conv2D) (None, 8, 8, 160) 179200 ['activation_150[0][0]']
batch_normalization_147 (Batch (None, 8, 8, 160) 480 ['conv2d_151[0][0]']
Normalization)
batch_normalization_152 (Batch (None, 8, 8, 160) 480 ['conv2d_156[0][0]']
Normalization)
activation_146 (Activation) (None, 8, 8, 160) 0 ['batch_normalization_147[0][0]']
activation_151 (Activation) (None, 8, 8, 160) 0 ['batch_normalization_152[0][0]']
average_pooling2d_14 (AverageP (None, 8, 8, 768) 0 ['mixed5[0][0]']
ooling2D)
conv2d_149 (Conv2D) (None, 8, 8, 192) 147456 ['mixed5[0][0]']
conv2d_152 (Conv2D) (None, 8, 8, 192) 215040 ['activation_146[0][0]']
conv2d_157 (Conv2D) (None, 8, 8, 192) 215040 ['activation_151[0][0]']
conv2d_158 (Conv2D) (None, 8, 8, 192) 147456 ['average_pooling2d_14[0][0]']
batch_normalization_145 (Batch (None, 8, 8, 192) 576 ['conv2d_149[0][0]']
Normalization)
batch_normalization_148 (Batch (None, 8, 8, 192) 576 ['conv2d_152[0][0]']
Normalization)
batch_normalization_153 (Batch (None, 8, 8, 192) 576 ['conv2d_157[0][0]']
Normalization)
batch_normalization_154 (Batch (None, 8, 8, 192) 576 ['conv2d_158[0][0]']
Normalization)
activation_144 (Activation) (None, 8, 8, 192) 0 ['batch_normalization_145[0][0]']
activation_147 (Activation) (None, 8, 8, 192) 0 ['batch_normalization_148[0][0]']
activation_152 (Activation) (None, 8, 8, 192) 0 ['batch_normalization_153[0][0]']
activation_153 (Activation) (None, 8, 8, 192) 0 ['batch_normalization_154[0][0]']
mixed6 (Concatenate) (None, 8, 8, 768) 0 ['activation_144[0][0]',
'activation_147[0][0]',
'activation_152[0][0]',
'activation_153[0][0]']
conv2d_163 (Conv2D) (None, 8, 8, 192) 147456 ['mixed6[0][0]']
batch_normalization_159 (Batch (None, 8, 8, 192) 576 ['conv2d_163[0][0]']
Normalization)
activation_158 (Activation) (None, 8, 8, 192) 0 ['batch_normalization_159[0][0]']
conv2d_164 (Conv2D) (None, 8, 8, 192) 258048 ['activation_158[0][0]']
batch_normalization_160 (Batch (None, 8, 8, 192) 576 ['conv2d_164[0][0]']
Normalization)
activation_159 (Activation) (None, 8, 8, 192) 0 ['batch_normalization_160[0][0]']
conv2d_160 (Conv2D) (None, 8, 8, 192) 147456 ['mixed6[0][0]']
conv2d_165 (Conv2D) (None, 8, 8, 192) 258048 ['activation_159[0][0]']
batch_normalization_156 (Batch (None, 8, 8, 192) 576 ['conv2d_160[0][0]']
Normalization)
batch_normalization_161 (Batch (None, 8, 8, 192) 576 ['conv2d_165[0][0]']
Normalization)
activation_155 (Activation) (None, 8, 8, 192) 0 ['batch_normalization_156[0][0]']
activation_160 (Activation) (None, 8, 8, 192) 0 ['batch_normalization_161[0][0]']
conv2d_161 (Conv2D) (None, 8, 8, 192) 258048 ['activation_155[0][0]']
conv2d_166 (Conv2D) (None, 8, 8, 192) 258048 ['activation_160[0][0]']
batch_normalization_157 (Batch (None, 8, 8, 192) 576 ['conv2d_161[0][0]']
Normalization)
batch_normalization_162 (Batch (None, 8, 8, 192) 576 ['conv2d_166[0][0]']
Normalization)
activation_156 (Activation) (None, 8, 8, 192) 0 ['batch_normalization_157[0][0]']
activation_161 (Activation) (None, 8, 8, 192) 0 ['batch_normalization_162[0][0]']
average_pooling2d_15 (AverageP (None, 8, 8, 768) 0 ['mixed6[0][0]']
ooling2D)
conv2d_159 (Conv2D) (None, 8, 8, 192) 147456 ['mixed6[0][0]']
conv2d_162 (Conv2D) (None, 8, 8, 192) 258048 ['activation_156[0][0]']
conv2d_167 (Conv2D) (None, 8, 8, 192) 258048 ['activation_161[0][0]']
conv2d_168 (Conv2D) (None, 8, 8, 192) 147456 ['average_pooling2d_15[0][0]']
batch_normalization_155 (Batch (None, 8, 8, 192) 576 ['conv2d_159[0][0]']
Normalization)
batch_normalization_158 (Batch (None, 8, 8, 192) 576 ['conv2d_162[0][0]']
Normalization)
batch_normalization_163 (Batch (None, 8, 8, 192) 576 ['conv2d_167[0][0]']
Normalization)
batch_normalization_164 (Batch (None, 8, 8, 192) 576 ['conv2d_168[0][0]']
Normalization)
activation_154 (Activation) (None, 8, 8, 192) 0 ['batch_normalization_155[0][0]']
activation_157 (Activation) (None, 8, 8, 192) 0 ['batch_normalization_158[0][0]']
activation_162 (Activation) (None, 8, 8, 192) 0 ['batch_normalization_163[0][0]']
activation_163 (Activation) (None, 8, 8, 192) 0 ['batch_normalization_164[0][0]']
mixed7 (Concatenate) (None, 8, 8, 768) 0 ['activation_154[0][0]',
'activation_157[0][0]',
'activation_162[0][0]',
'activation_163[0][0]']
conv2d_171 (Conv2D) (None, 8, 8, 192) 147456 ['mixed7[0][0]']
batch_normalization_167 (Batch (None, 8, 8, 192) 576 ['conv2d_171[0][0]']
Normalization)
activation_166 (Activation) (None, 8, 8, 192) 0 ['batch_normalization_167[0][0]']
conv2d_172 (Conv2D) (None, 8, 8, 192) 258048 ['activation_166[0][0]']
batch_normalization_168 (Batch (None, 8, 8, 192) 576 ['conv2d_172[0][0]']
Normalization)
activation_167 (Activation) (None, 8, 8, 192) 0 ['batch_normalization_168[0][0]']
conv2d_169 (Conv2D) (None, 8, 8, 192) 147456 ['mixed7[0][0]']
conv2d_173 (Conv2D) (None, 8, 8, 192) 258048 ['activation_167[0][0]']
batch_normalization_165 (Batch (None, 8, 8, 192) 576 ['conv2d_169[0][0]']
Normalization)
batch_normalization_169 (Batch (None, 8, 8, 192) 576 ['conv2d_173[0][0]']
Normalization)
activation_164 (Activation) (None, 8, 8, 192) 0 ['batch_normalization_165[0][0]']
activation_168 (Activation) (None, 8, 8, 192) 0 ['batch_normalization_169[0][0]']
conv2d_170 (Conv2D) (None, 4, 4, 320) 552960 ['activation_164[0][0]']
conv2d_174 (Conv2D) (None, 4, 4, 192) 331776 ['activation_168[0][0]']
batch_normalization_166 (Batch (None, 4, 4, 320) 960 ['conv2d_170[0][0]']
Normalization)
batch_normalization_170 (Batch (None, 4, 4, 192) 576 ['conv2d_174[0][0]']
Normalization)
activation_165 (Activation) (None, 4, 4, 320) 0 ['batch_normalization_166[0][0]']
activation_169 (Activation) (None, 4, 4, 192) 0 ['batch_normalization_170[0][0]']
max_pooling2d_11 (MaxPooling2D (None, 4, 4, 768) 0 ['mixed7[0][0]']
)
mixed8 (Concatenate) (None, 4, 4, 1280) 0 ['activation_165[0][0]',
'activation_169[0][0]',
'max_pooling2d_11[0][0]']
conv2d_179 (Conv2D) (None, 4, 4, 448) 573440 ['mixed8[0][0]']
batch_normalization_175 (Batch (None, 4, 4, 448) 1344 ['conv2d_179[0][0]']
Normalization)
activation_174 (Activation) (None, 4, 4, 448) 0 ['batch_normalization_175[0][0]']
conv2d_176 (Conv2D) (None, 4, 4, 384) 491520 ['mixed8[0][0]']
conv2d_180 (Conv2D) (None, 4, 4, 384) 1548288 ['activation_174[0][0]']
batch_normalization_172 (Batch (None, 4, 4, 384) 1152 ['conv2d_176[0][0]']
Normalization)
batch_normalization_176 (Batch (None, 4, 4, 384) 1152 ['conv2d_180[0][0]']
Normalization)
activation_171 (Activation) (None, 4, 4, 384) 0 ['batch_normalization_172[0][0]']
activation_175 (Activation) (None, 4, 4, 384) 0 ['batch_normalization_176[0][0]']
conv2d_177 (Conv2D) (None, 4, 4, 384) 442368 ['activation_171[0][0]']
conv2d_178 (Conv2D) (None, 4, 4, 384) 442368 ['activation_171[0][0]']
conv2d_181 (Conv2D) (None, 4, 4, 384) 442368 ['activation_175[0][0]']
conv2d_182 (Conv2D) (None, 4, 4, 384) 442368 ['activation_175[0][0]']
average_pooling2d_16 (AverageP (None, 4, 4, 1280) 0 ['mixed8[0][0]']
ooling2D)
conv2d_175 (Conv2D) (None, 4, 4, 320) 409600 ['mixed8[0][0]']
batch_normalization_173 (Batch (None, 4, 4, 384) 1152 ['conv2d_177[0][0]']
Normalization)
batch_normalization_174 (Batch (None, 4, 4, 384) 1152 ['conv2d_178[0][0]']
Normalization)
batch_normalization_177 (Batch (None, 4, 4, 384) 1152 ['conv2d_181[0][0]']
Normalization)
batch_normalization_178 (Batch (None, 4, 4, 384) 1152 ['conv2d_182[0][0]']
Normalization)
conv2d_183 (Conv2D) (None, 4, 4, 192) 245760 ['average_pooling2d_16[0][0]']
batch_normalization_171 (Batch (None, 4, 4, 320) 960 ['conv2d_175[0][0]']
Normalization)
activation_172 (Activation) (None, 4, 4, 384) 0 ['batch_normalization_173[0][0]']
activation_173 (Activation) (None, 4, 4, 384) 0 ['batch_normalization_174[0][0]']
activation_176 (Activation) (None, 4, 4, 384) 0 ['batch_normalization_177[0][0]']
activation_177 (Activation) (None, 4, 4, 384) 0 ['batch_normalization_178[0][0]']
batch_normalization_179 (Batch (None, 4, 4, 192) 576 ['conv2d_183[0][0]']
Normalization)
activation_170 (Activation) (None, 4, 4, 320) 0 ['batch_normalization_171[0][0]']
mixed9_0 (Concatenate) (None, 4, 4, 768) 0 ['activation_172[0][0]',
'activation_173[0][0]']
concatenate_7 (Concatenate) (None, 4, 4, 768) 0 ['activation_176[0][0]',
'activation_177[0][0]']
activation_178 (Activation) (None, 4, 4, 192) 0 ['batch_normalization_179[0][0]']
mixed9 (Concatenate) (None, 4, 4, 2048) 0 ['activation_170[0][0]',
'mixed9_0[0][0]',
'concatenate_7[0][0]',
'activation_178[0][0]']
conv2d_188 (Conv2D) (None, 4, 4, 448) 917504 ['mixed9[0][0]']
batch_normalization_184 (Batch (None, 4, 4, 448) 1344 ['conv2d_188[0][0]']
Normalization)
activation_183 (Activation) (None, 4, 4, 448) 0 ['batch_normalization_184[0][0]']
conv2d_185 (Conv2D) (None, 4, 4, 384) 786432 ['mixed9[0][0]']
conv2d_189 (Conv2D) (None, 4, 4, 384) 1548288 ['activation_183[0][0]']
batch_normalization_181 (Batch (None, 4, 4, 384) 1152 ['conv2d_185[0][0]']
Normalization)
batch_normalization_185 (Batch (None, 4, 4, 384) 1152 ['conv2d_189[0][0]']
Normalization)
activation_180 (Activation) (None, 4, 4, 384) 0 ['batch_normalization_181[0][0]']
activation_184 (Activation) (None, 4, 4, 384) 0 ['batch_normalization_185[0][0]']
conv2d_186 (Conv2D) (None, 4, 4, 384) 442368 ['activation_180[0][0]']
conv2d_187 (Conv2D) (None, 4, 4, 384) 442368 ['activation_180[0][0]']
conv2d_190 (Conv2D) (None, 4, 4, 384) 442368 ['activation_184[0][0]']
conv2d_191 (Conv2D) (None, 4, 4, 384) 442368 ['activation_184[0][0]']
average_pooling2d_17 (AverageP (None, 4, 4, 2048) 0 ['mixed9[0][0]']
ooling2D)
conv2d_184 (Conv2D) (None, 4, 4, 320) 655360 ['mixed9[0][0]']
batch_normalization_182 (Batch (None, 4, 4, 384) 1152 ['conv2d_186[0][0]']
Normalization)
batch_normalization_183 (Batch (None, 4, 4, 384) 1152 ['conv2d_187[0][0]']
Normalization)
batch_normalization_186 (Batch (None, 4, 4, 384) 1152 ['conv2d_190[0][0]']
Normalization)
batch_normalization_187 (Batch (None, 4, 4, 384) 1152 ['conv2d_191[0][0]']
Normalization)
conv2d_192 (Conv2D) (None, 4, 4, 192) 393216 ['average_pooling2d_17[0][0]']
batch_normalization_180 (Batch (None, 4, 4, 320) 960 ['conv2d_184[0][0]']
Normalization)
activation_181 (Activation) (None, 4, 4, 384) 0 ['batch_normalization_182[0][0]']
activation_182 (Activation) (None, 4, 4, 384) 0 ['batch_normalization_183[0][0]']
activation_185 (Activation) (None, 4, 4, 384) 0 ['batch_normalization_186[0][0]']
activation_186 (Activation) (None, 4, 4, 384) 0 ['batch_normalization_187[0][0]']
batch_normalization_188 (Batch (None, 4, 4, 192) 576 ['conv2d_192[0][0]']
Normalization)
activation_179 (Activation) (None, 4, 4, 320) 0 ['batch_normalization_180[0][0]']
mixed9_1 (Concatenate) (None, 4, 4, 768) 0 ['activation_181[0][0]',
'activation_182[0][0]']
concatenate_8 (Concatenate) (None, 4, 4, 768) 0 ['activation_185[0][0]',
'activation_186[0][0]']
activation_187 (Activation) (None, 4, 4, 192) 0 ['batch_normalization_188[0][0]']
mixed10 (Concatenate) (None, 4, 4, 2048) 0 ['activation_179[0][0]',
'mixed9_1[0][0]',
'concatenate_8[0][0]',
'activation_187[0][0]']
decoder_stage0_upsampling (UpS (None, 8, 8, 2048) 0 ['mixed10[0][0]']
ampling2D)
decoder_stage0_concat (Concate (None, 8, 8, 2816) 0 ['decoder_stage0_upsampling[0][0]
nate) ',
'mixed7[0][0]']
decoder_stage0a_conv (Conv2D) (None, 8, 8, 256) 6488064 ['decoder_stage0_concat[0][0]']
decoder_stage0a_bn (BatchNorma (None, 8, 8, 256) 1024 ['decoder_stage0a_conv[0][0]']
lization)
decoder_stage0a_relu (Activati (None, 8, 8, 256) 0 ['decoder_stage0a_bn[0][0]']
on)
decoder_stage0b_conv (Conv2D) (None, 8, 8, 256) 589824 ['decoder_stage0a_relu[0][0]']
decoder_stage0b_bn (BatchNorma (None, 8, 8, 256) 1024 ['decoder_stage0b_conv[0][0]']
lization)
decoder_stage0b_relu (Activati (None, 8, 8, 256) 0 ['decoder_stage0b_bn[0][0]']
on)
decoder_stage1_upsampling (UpS (None, 16, 16, 256) 0 ['decoder_stage0b_relu[0][0]']
ampling2D)
decoder_stage1_concat (Concate (None, 16, 16, 544) 0 ['decoder_stage1_upsampling[0][0]
nate) ',
'mixed2[0][0]']
decoder_stage1a_conv (Conv2D) (None, 16, 16, 128) 626688 ['decoder_stage1_concat[0][0]']
decoder_stage1a_bn (BatchNorma (None, 16, 16, 128) 512 ['decoder_stage1a_conv[0][0]']
lization)
decoder_stage1a_relu (Activati (None, 16, 16, 128) 0 ['decoder_stage1a_bn[0][0]']
on)
decoder_stage1b_conv (Conv2D) (None, 16, 16, 128) 147456 ['decoder_stage1a_relu[0][0]']
decoder_stage1b_bn (BatchNorma (None, 16, 16, 128) 512 ['decoder_stage1b_conv[0][0]']
lization)
decoder_stage1b_relu (Activati (None, 16, 16, 128) 0 ['decoder_stage1b_bn[0][0]']
on)
decoder_stage2_upsampling (UpS (None, 32, 32, 128) 0 ['decoder_stage1b_relu[0][0]']
ampling2D)
decoder_stage2_concat (Concate (None, 32, 32, 320) 0 ['decoder_stage2_upsampling[0][0]
nate) ',
'activation_98[0][0]']
decoder_stage2a_conv (Conv2D) (None, 32, 32, 64) 184320 ['decoder_stage2_concat[0][0]']
decoder_stage2a_bn (BatchNorma (None, 32, 32, 64) 256 ['decoder_stage2a_conv[0][0]']
lization)
decoder_stage2a_relu (Activati (None, 32, 32, 64) 0 ['decoder_stage2a_bn[0][0]']
on)
decoder_stage2b_conv (Conv2D) (None, 32, 32, 64) 36864 ['decoder_stage2a_relu[0][0]']
decoder_stage2b_bn (BatchNorma (None, 32, 32, 64) 256 ['decoder_stage2b_conv[0][0]']
lization)
decoder_stage2b_relu (Activati (None, 32, 32, 64) 0 ['decoder_stage2b_bn[0][0]']
on)
decoder_stage3_upsampling (UpS (None, 64, 64, 64) 0 ['decoder_stage2b_relu[0][0]']
ampling2D)
decoder_stage3_concat (Concate (None, 64, 64, 128) 0 ['decoder_stage3_upsampling[0][0]
nate) ',
'activation_96[0][0]']
decoder_stage3a_conv (Conv2D) (None, 64, 64, 32) 36864 ['decoder_stage3_concat[0][0]']
decoder_stage3a_bn (BatchNorma (None, 64, 64, 32) 128 ['decoder_stage3a_conv[0][0]']
lization)
decoder_stage3a_relu (Activati (None, 64, 64, 32) 0 ['decoder_stage3a_bn[0][0]']
on)
decoder_stage3b_conv (Conv2D) (None, 64, 64, 32) 9216 ['decoder_stage3a_relu[0][0]']
decoder_stage3b_bn (BatchNorma (None, 64, 64, 32) 128 ['decoder_stage3b_conv[0][0]']
lization)
decoder_stage3b_relu (Activati (None, 64, 64, 32) 0 ['decoder_stage3b_bn[0][0]']
on)
decoder_stage4_upsampling (UpS (None, 128, 128, 32 0 ['decoder_stage3b_relu[0][0]']
ampling2D) )
decoder_stage4a_conv (Conv2D) (None, 128, 128, 16 4608 ['decoder_stage4_upsampling[0][0]
) ']
decoder_stage4a_bn (BatchNorma (None, 128, 128, 16 64 ['decoder_stage4a_conv[0][0]']
lization) )
decoder_stage4a_relu (Activati (None, 128, 128, 16 0 ['decoder_stage4a_bn[0][0]']
on) )
decoder_stage4b_conv (Conv2D) (None, 128, 128, 16 2304 ['decoder_stage4a_relu[0][0]']
)
decoder_stage4b_bn (BatchNorma (None, 128, 128, 16 64 ['decoder_stage4b_conv[0][0]']
lization) )
decoder_stage4b_relu (Activati (None, 128, 128, 16 0 ['decoder_stage4b_bn[0][0]']
on) )
final_conv (Conv2D) (None, 128, 128, 1) 145 ['decoder_stage4b_relu[0][0]']
sigmoid (Activation) (None, 128, 128, 1) 0 ['final_conv[0][0]']
==================================================================================================
Total params: 29,933,105
Trainable params: 29,896,689
Non-trainable params: 36,416
__________________________________________________________________________________________________
# Plot the model
tf.keras.utils.plot_model(model_unet_3, show_shapes=True)
# Fit the model
history_unet_3 = model_unet_3.fit(Train_X_unet_3, Train_mask_unet, epochs=20, batch_size=32, validation_split=0.35, callbacks=[early_stop, reduce_lr])
Epoch 1/20 69/69 [==============================] - 32s 309ms/step - loss: 1.4180 - dice_coefficient: 0.4186 - val_loss: 2.1286 - val_dice_coefficient: 0.4065 - lr: 1.0000e-04 Epoch 2/20 69/69 [==============================] - 16s 233ms/step - loss: 0.9964 - dice_coefficient: 0.5304 - val_loss: 1.4711 - val_dice_coefficient: 0.4800 - lr: 1.0000e-04 Epoch 3/20 69/69 [==============================] - 16s 233ms/step - loss: 0.8042 - dice_coefficient: 0.5908 - val_loss: 1.1162 - val_dice_coefficient: 0.5430 - lr: 1.0000e-04 Epoch 4/20 69/69 [==============================] - 16s 234ms/step - loss: 0.6690 - dice_coefficient: 0.6427 - val_loss: 1.0249 - val_dice_coefficient: 0.5727 - lr: 1.0000e-04 Epoch 5/20 69/69 [==============================] - 16s 234ms/step - loss: 0.5715 - dice_coefficient: 0.6854 - val_loss: 0.8412 - val_dice_coefficient: 0.6141 - lr: 1.0000e-04 Epoch 6/20 69/69 [==============================] - 16s 234ms/step - loss: 0.4961 - dice_coefficient: 0.7216 - val_loss: 0.7351 - val_dice_coefficient: 0.6328 - lr: 1.0000e-04 Epoch 7/20 69/69 [==============================] - 16s 235ms/step - loss: 0.4358 - dice_coefficient: 0.7524 - val_loss: 0.7119 - val_dice_coefficient: 0.6444 - lr: 1.0000e-04 Epoch 8/20 69/69 [==============================] - 16s 236ms/step - loss: 0.3861 - dice_coefficient: 0.7780 - val_loss: 0.7248 - val_dice_coefficient: 0.6453 - lr: 1.0000e-04 Epoch 9/20 69/69 [==============================] - 16s 236ms/step - loss: 0.3451 - dice_coefficient: 0.8003 - val_loss: 0.6765 - val_dice_coefficient: 0.6800 - lr: 1.0000e-04 Epoch 10/20 69/69 [==============================] - 16s 236ms/step - loss: 0.3131 - dice_coefficient: 0.8184 - val_loss: 0.6976 - val_dice_coefficient: 0.6645 - lr: 1.0000e-04 Epoch 11/20 69/69 [==============================] - 16s 236ms/step - loss: 0.2818 - dice_coefficient: 0.8353 - val_loss: 0.6929 - val_dice_coefficient: 0.6647 - lr: 1.0000e-04 Epoch 12/20 69/69 [==============================] - 16s 236ms/step - loss: 0.2681 - dice_coefficient: 0.8441 - val_loss: 0.7239 - val_dice_coefficient: 0.6582 - lr: 1.0000e-04 Epoch 13/20 69/69 [==============================] - 16s 236ms/step - loss: 0.2412 - dice_coefficient: 0.8586 - val_loss: 0.6685 - val_dice_coefficient: 0.6994 - lr: 1.0000e-04 Epoch 14/20 69/69 [==============================] - 16s 236ms/step - loss: 0.2187 - dice_coefficient: 0.8710 - val_loss: 0.6930 - val_dice_coefficient: 0.6846 - lr: 1.0000e-04 Epoch 15/20 69/69 [==============================] - 16s 236ms/step - loss: 0.2027 - dice_coefficient: 0.8803 - val_loss: 0.6816 - val_dice_coefficient: 0.6935 - lr: 1.0000e-04 Epoch 16/20 69/69 [==============================] - 16s 236ms/step - loss: 0.1915 - dice_coefficient: 0.8873 - val_loss: 0.6939 - val_dice_coefficient: 0.6898 - lr: 1.0000e-04 Epoch 17/20 69/69 [==============================] - 16s 236ms/step - loss: 0.1779 - dice_coefficient: 0.8949 - val_loss: 0.6719 - val_dice_coefficient: 0.7030 - lr: 1.0000e-04 Epoch 18/20 69/69 [==============================] - 16s 236ms/step - loss: 0.1720 - dice_coefficient: 0.8989 - val_loss: 0.7194 - val_dice_coefficient: 0.6809 - lr: 1.0000e-04 Epoch 19/20 69/69 [==============================] - 16s 236ms/step - loss: 0.1633 - dice_coefficient: 0.9040 - val_loss: 0.7964 - val_dice_coefficient: 0.6543 - lr: 1.0000e-04 Epoch 20/20 69/69 [==============================] - 16s 236ms/step - loss: 0.1561 - dice_coefficient: 0.9081 - val_loss: 0.7206 - val_dice_coefficient: 0.6951 - lr: 1.0000e-04
# Plot accuracy and loss
plot_accuracy_loss(history=history_unet_3, model_type='UNET')
# Model Evaluation
eval_acc_unet_3, eval_loss_unet_3, train_acc_unet_3, train_loss_unet_3 = evaluate_model(model_unet_3, Eval_X_unet_3, Eval_mask_unet, Train_X_unet_3, Train_mask_unet, 'UNET')
1/1 [==============================] - 1s 983ms/step - loss: 0.6334 - dice_coefficient: 0.7133 106/106 [==============================] - 8s 75ms/step - loss: 0.3479 - dice_coefficient: 0.8347 Dice Coefficient on Eval set = 0.7133 & Loss on Eval set = 0.6334
# Append the scores & loss
unet_model_scores['Model'].append('InceptionV3')
unet_model_scores['Train Score'].append(train_acc_unet_3)
unet_model_scores['Eval Score'].append(eval_acc_unet_3)
unet_model_scores['Train Loss'].append(train_loss_unet_3)
unet_model_scores['Eval Loss'].append(eval_loss_unet_3)
del Train_X_unet_3
del Eval_X_unet_3
# Convert dictionary into DataFrame
unet_models_df = pd.DataFrame(unet_model_scores)
unet_models_df
| Model | Train Score | Eval Score | Train Loss | Eval Loss | |
|---|---|---|---|---|---|
| 0 | MobileNet | 0.6470 | 0.6747 | 0.6790 | 0.6187 |
| 1 | Resnet34 | 0.8346 | 0.6461 | 0.3443 | 0.8286 |
| 2 | InceptionV3 | 0.8347 | 0.7133 | 0.3479 | 0.6334 |
# Plot the accuracy and loss for all unet models
plt.figure(figsize=(20,6))
plt.subplot(1, 2, 1)
sns.lineplot(x=unet_models_df.Model, y=unet_models_df['Train Score'], marker='o', label='Train Score')
sns.lineplot(x=unet_models_df.Model, y=unet_models_df['Eval Score'], marker='o', label='Eval Score', linestyle='--')
plt.xlabel('Models')
plt.ylabel('Dice Coefficient')
plt.title('Model Accuracy')
plt.subplot(1, 2, 2)
sns.lineplot(x=unet_models_df.Model, y=unet_models_df['Train Loss'], marker='s', label='Train Loss')
sns.lineplot(x=unet_models_df.Model, y=unet_models_df['Eval Loss'], marker='s', label='Eval Loss', linestyle='--')
plt.xlabel('Models')
plt.ylabel('Loss')
plt.title('Model Loss')
plt.show()
os.chdir('/content/drive/MyDrive/Colab Notebooks')
# Save the best models
model_clf_2.save('model_clf.h5')
model_unet_3.save('model_unet.h5')
os.chdir('/content/drive/MyDrive/Colab Notebooks')
# Load the saved models
final_model_clf = tf.keras.models.load_model('model_clf.h5', compile=False)
final_model_unet = tf.keras.models.load_model('model_unet.h5', compile=False)
os.chdir('/content')
# Constants for the below functions
IMAGE_WIDTH = 128
IMAGE_HEIGHT = 128
RATIO = 1024/128
THRESHOLD = 0.5
# Function to plot a single predicted image
def plot_single_test_image_prediction(df, image):
plt.figure(figsize=(10,6))
if(df.Target[0] == 1):
plt.title('PatientID - ' + df.PatientID[0] + '\nPredicted Label - Pneumonia')
else:
plt.title('PatientID - ' + df.PatientID[0] + '\nPredicted Label - Non Pneumonia')
plt.imshow(image)
plt.axis('Off')
plt.show()
# Function to predict single image
def single_test_image_prediction(path, model_clf, model_unet, preprocess, width, height):
points = []
list_points = []
col = ['X1','Y1','X2','Y2','Area'] # Columns for contours
im = dicom.dcmread(path) # Read Dicom image
img_arr_org = im.pixel_array
img_arr_org = np.stack([img_arr_org]*3, axis=2) # Convert to 3 channels
img_arr_copy = np.copy(img_arr_org)
patient_id = im.PatientID # Get PatientID
orig_im_h, orig_im_w, channels = img_arr_org.shape
img_arr_resize = cv2.resize(img_arr_copy, (width, height)) # Resize the image
pp_img_arr = preprocess(np.array(img_arr_resize, dtype=np.float32)) # Apply pre-process function
pp_img_arr_ = np.expand_dims(pp_img_arr, axis=0)
predict_vec = model_clf.predict(pp_img_arr_) # Predict Target
class_predicted = np.argmax(predict_vec)
if(class_predicted == 1):
pred_mask = model_unet.predict(pp_img_arr_) # Predict Bounding boxes
output = np.zeros(pp_img_arr.shape[:2], dtype=np.uint8)
for i in range(pred_mask[0].shape[1]):
for j in range(pred_mask[0].shape[0]):
if pred_mask[0][i][j] > THRESHOLD:
x1 = int(j * pp_img_arr.shape[1] / IMAGE_WIDTH)
y1 = int(i * pp_img_arr.shape[0] / IMAGE_HEIGHT)
x2 = int((j + 1) * pp_img_arr.shape[1] / IMAGE_WIDTH)
y2 = int((i + 1) * pp_img_arr.shape[0] / IMAGE_HEIGHT)
output[y1:y2, x1:x2] = 1
contours,_ = cv2.findContours(output, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE) # Finding all contours
for cnt in contours:
approx = cv2.approxPolyDP(cnt, 0.02*cv2.arcLength(cnt, True), True)
x, y, w, h = cv2.boundingRect(approx)
points.append(x)
points.append(y)
points.append(x+w)
points.append(y+h)
points.append(w*h)
list_points.append(list(points))
points=[]
df = pd.DataFrame(list_points, columns=col)
df.sort_values(by='Area', inplace=True, ascending=False) # Sorting the area of bounding boxes
list_points = []
df = df[:2] # Taking 2 sets of co-ordinates for 2 bounding boxes
if(df.shape[0] == 2):
x11, y11, x12, y12, _ = np.array(RATIO*np.array(df[:1])[0], dtype=np.int64)
x21, y21, x22, y22, _ = np.array(RATIO*np.array(df[1:])[0], dtype=np.int64)
img_arr_org = cv2.rectangle(img_arr_org, (x11,y11), (x12,y12), (220, 20, 60), 5) # Creating bounding box 1
img_arr_org = cv2.rectangle(img_arr_org, (x21,y21), (x22,y22), (220, 20, 60), 5) # Creating bounding box 2
# Dataframe to return
pred_df = pd.DataFrame([[patient_id, x11, y11, x12, y12, x21, y21, x22, y22, class_predicted, round(predict_vec[0][1],2)]], columns=['PatientID','B1X1','B1Y1','B1X2','B1Y2','B2X1','B2Y1','B2X2','B2Y2','Target','Confidence'])
elif(df.shape[0] == 1):
x11, y11, x12, y12, _ = np.array(RATIO*np.array(df[:1])[0], dtype=np.int64)
img_arr_org = cv2.rectangle(img_arr_org, (x11,y11), (x12,y12), (220, 20, 60), 5) # Creating bounding box 1
pred_df = pd.DataFrame([[patient_id, x11, y11, x12, y12, 0, 0, 0, 0, class_predicted, round(predict_vec[0][1],2)]], columns=['PatientID','B1X1','B1Y1','B1X2','B1Y2','B2X1','B2Y1','B2X2','B2Y2','Target','Confidence'])
else:
pred_df = pd.DataFrame([[patient_id, 0, 0, 0, 0, 0, 0, 0, 0, 0, round(predict_vec[0][1],2)]], columns=['PatientID','B1X1','B1Y1','B1X2','B1Y2','B2X1','B2Y1','B2X2','B2Y2','Target','Confidence'])
else:
# Dataframe to return
pred_df = pd.DataFrame([[patient_id, 0, 0, 0, 0, 0, 0, 0, 0, 0, round(predict_vec[0][1],2)]], columns=['PatientID','B1X1','B1Y1','B1X2','B1Y2','B2X1','B2Y1','B2X2','B2Y2','Target','Confidence'])
return pred_df, img_arr_org
# Predicting single image from test folder which is likely to be Pneumonia
single_test_df, image = single_test_image_prediction(
path = '/content/stage_2_test_images/139086b8-d4bf-4ee5-8027-cb405da2c2ab.dcm',
model_clf = final_model_clf,
model_unet = final_model_unet,
preprocess = tf.keras.applications.mobilenet.preprocess_input,
width = 128,
height = 128)
plot_single_test_image_prediction(single_test_df, image) # Show the output image
single_test_df # Display the DataFrame
| PatientID | B1X1 | B1Y1 | B1X2 | B1Y2 | B2X1 | B2Y1 | B2X2 | B2Y2 | Target | Confidence | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 139086b8-d4bf-4ee5-8027-cb405da2c2ab | 168 | 400 | 384 | 688 | 592 | 408 | 848 | 632 | 1 | 0.74 |
# Predicting single image from test folder which is likely to be Non Pneumonia
single_test_df, image = single_test_image_prediction(
path = '/content/stage_2_test_images/2704fcb3-7b66-49ed-a5ba-ea20386c824a.dcm',
model_clf = final_model_clf,
model_unet = final_model_unet,
preprocess = tf.keras.applications.mobilenet.preprocess_input,
width = 128,
height = 128)
plot_single_test_image_prediction(single_test_df, image) # Show the output image
single_test_df # Display the DataFrame
| PatientID | B1X1 | B1Y1 | B1X2 | B1Y2 | B2X1 | B2Y1 | B2X2 | B2Y2 | Target | Confidence | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 2704fcb3-7b66-49ed-a5ba-ea20386c824a | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.0 |
# Function to predict all the images in the given path
def bulk_test_image_prediction(path, model_clf, model_unet, preprocess, width, height):
points = []
list_points = []
col = ['X1','Y1','X2','Y2','Area'] # Columns for contours
pred_df = pd.DataFrame(columns=['PatientID','B1X1','B1Y1','B1X2','B1Y2','B2X1','B2Y1','B2X2','B2Y2','Target','Confidence'])
os.chdir(path)
folder = os.listdir()
for im in folder:
img = dicom.dcmread(im) # Read Dicom image
img_arr_org = img.pixel_array
img_arr_org = np.stack([img_arr_org]*3, axis=2) # Convert to 3 channels
img_arr_copy = np.copy(img_arr_org)
patient_id = img.PatientID # Get PatientID
orig_im_h, orig_im_w, channels = img_arr_org.shape
img_arr_resize = cv2.resize(img_arr_copy, (width, height)) # Resize the image
pp_img_arr = preprocess(np.array(img_arr_resize, dtype=np.float32)) # Apply pre-process function
pp_img_arr_ = np.expand_dims(pp_img_arr, axis=0)
predict_vec = model_clf.predict(pp_img_arr_) # Predict Target
class_predicted = np.argmax(predict_vec)
if(class_predicted == 1):
pred_mask = model_unet.predict(pp_img_arr_) # Predict Bounding boxes
output = np.zeros(pp_img_arr.shape[:2], dtype=np.uint8)
for i in range(pred_mask[0].shape[1]):
for j in range(pred_mask[0].shape[0]):
if pred_mask[0][i][j] > THRESHOLD:
x1 = int(j * pp_img_arr.shape[1] / IMAGE_WIDTH)
y1 = int(i * pp_img_arr.shape[0] / IMAGE_HEIGHT)
x2 = int((j + 1) * pp_img_arr.shape[1] / IMAGE_WIDTH)
y2 = int((i + 1) * pp_img_arr.shape[0] / IMAGE_HEIGHT)
output[y1:y2, x1:x2] = 1
contours,_ = cv2.findContours(output, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE) # Finding all contours
for cnt in contours:
approx = cv2.approxPolyDP(cnt, 0.02*cv2.arcLength(cnt, True), True)
x, y, w, h = cv2.boundingRect(approx)
points.append(x)
points.append(y)
points.append(x+w)
points.append(y+h)
points.append(w*h)
list_points.append(list(points))
points=[]
df = pd.DataFrame(list_points, columns=col)
df.sort_values(by='Area', inplace=True, ascending=False) # Sorting the area of bounding boxes
list_points = []
df = df[:2] # Taking 2 sets of co-ordinates for 2 bounding boxes
if(df.shape[0] == 2):
x11, y11, x12, y12, _ = np.array(RATIO*np.array(df[:1])[0], dtype=np.int64) # Co-ordinates for bounding box 1
x21, y21, x22, y22, _ = np.array(RATIO*np.array(df[1:])[0], dtype=np.int64) # Co-ordinates for bounding box 2
pred_1 = pd.DataFrame([[patient_id, x11, y11, x12, y12, x21, y21, x22, y22, class_predicted, round(predict_vec[0][1],2)]], columns=['PatientID','B1X1','B1Y1','B1X2','B1Y2','B2X1','B2Y1','B2X2','B2Y2','Target','Confidence'])
# Dataframe to return
pred_df = pd.concat([pred_df, pred_1], ignore_index=True)
elif(df.shape[0] == 1):
x11, y11, x12, y12, _ = np.array(RATIO*np.array(df[:1])[0], dtype=np.int64)
pred_1 = pd.DataFrame([[patient_id, x11, y11, x12, y12, 0, 0, 0, 0, class_predicted, round(predict_vec[0][1],2)]], columns=['PatientID','B1X1','B1Y1','B1X2','B1Y2','B2X1','B2Y1','B2X2','B2Y2','Target','Confidence'])
pred_df = pd.concat([pred_df, pred_1], ignore_index=True)
else:
pred_1 = pd.DataFrame([[patient_id, 0, 0, 0, 0, 0, 0, 0, 0, 0, round(predict_vec[0][1],2)]], columns=['PatientID','B1X1','B1Y1','B1X2','B1Y2','B2X1','B2Y1','B2X2','B2Y2','Target','Confidence'])
pred_df = pd.concat([pred_df, pred_1], ignore_index=True)
else:
pred_0 = pd.DataFrame([[patient_id, 0, 0, 0, 0, 0, 0, 0, 0, 0, round(predict_vec[0][1],2)]], columns=['PatientID','B1X1','B1Y1','B1X2','B1Y2','B2X1','B2Y1','B2X2','B2Y2','Target','Confidence'])
# Dataframe to return
pred_df = pd.concat([pred_df, pred_0], ignore_index=True)
return pred_df
# Predicting all the images from test folder
bulk_test_image_prediction(
path = '/content/stage_2_test_images',
model_clf = final_model_clf,
model_unet = final_model_unet,
preprocess = tf.keras.applications.mobilenet.preprocess_input,
width = 128,
height = 128)
| PatientID | B1X1 | B1Y1 | B1X2 | B1Y2 | B2X1 | B2Y1 | B2X2 | B2Y2 | Target | Confidence | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 048a68fe-87f7-44a2-953d-d00651cc2605 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.47 |
| 1 | 1cec3d93-889c-4f4f-ae28-72635b933694 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.25 |
| 2 | 027301a0-cb61-4431-8736-503d0c23cd48 | 432 | 168 | 696 | 664 | 24 | 152 | 280 | 656 | 1 | 0.71 |
| 3 | 10cdb6ec-b1e9-45ec-a3f3-2364dd30346b | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.27 |
| 4 | 2287cc8e-e51d-4e32-9eca-760c49e4539d | 504 | 464 | 672 | 848 | 240 | 416 | 408 | 792 | 1 | 0.93 |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 2995 | 244330f7-5f3e-4c90-a20f-2e59e78c3b12 | 672 | 400 | 848 | 776 | 280 | 544 | 440 | 808 | 1 | 0.76 |
| 2996 | 11d313d4-1d96-4da8-8fe0-8a0db9910d04 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.32 |
| 2997 | 11f3fd57-b8c5-4345-8a32-d30b3316bf40 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.30 |
| 2998 | 2feb78d1-e0e0-42d5-a7a8-ef42e655bcac | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.04 |
| 2999 | 1db96f41-f271-42f7-969f-025733684776 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0.20 |
3000 rows × 11 columns
# Converting Dicom test images into Jpg to display in the Gradio UI
os.chdir('/content')
folders = ['stage_2_test_images']
for folder in folders:
folder_path = folder
# Check if folder exists
if os.path.exists('JPG.'+folder):
break
else:
jpg_folder_path = 'JPG.'+folder
os.mkdir(jpg_folder_path) # Create folder with prefix 'JPG.'
images_path = os.listdir(folder_path)
max_num=len(images_path)
print('Total Images in {0} : {1}'.format(folder, max_num))
for n, image in enumerate(images_path):
ds = dicom.dcmread(os.path.join(folder_path, image))
pixel_array_numpy = ds.pixel_array
image = image.replace('.dcm', '.jpg')
cv2.imwrite(os.path.join(jpg_folder_path, image), pixel_array_numpy)
if n % 1000 == 0 and n != 0 :
print('{0} images converted'.format(n))
elif n == max_num-1 and n != 0:
print('{0} images converted'.format(max_num))
print('Successfully converted all images to Jpg.')
Total Images in stage_2_test_images : 3000 1000 images converted 2000 images converted 3000 images converted Successfully converted all images to Jpg.
# Install gradio
!pip install gradio
Collecting gradio
Downloading gradio-2.9.4-py3-none-any.whl (2.9 MB)
|████████████████████████████████| 2.9 MB 4.2 MB/s
Collecting orjson
Downloading orjson-3.6.8-cp37-cp37m-manylinux_2_24_x86_64.whl (253 kB)
|████████████████████████████████| 253 kB 71.2 MB/s
Requirement already satisfied: Jinja2 in /usr/local/lib/python3.7/dist-packages (from gradio) (2.11.3)
Collecting pycryptodome
Downloading pycryptodome-3.14.1-cp35-abi3-manylinux2010_x86_64.whl (2.0 MB)
|████████████████████████████████| 2.0 MB 49.2 MB/s
Collecting pydub
Downloading pydub-0.25.1-py2.py3-none-any.whl (32 kB)
Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from gradio) (2.23.0)
Collecting ffmpy
Downloading ffmpy-0.3.0.tar.gz (4.8 kB)
Requirement already satisfied: matplotlib in /usr/local/lib/python3.7/dist-packages (from gradio) (3.2.2)
Collecting fastapi
Downloading fastapi-0.76.0-py3-none-any.whl (54 kB)
|████████████████████████████████| 54 kB 3.6 MB/s
Collecting aiohttp
Downloading aiohttp-3.8.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.1 MB)
|████████████████████████████████| 1.1 MB 51.5 MB/s
Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from gradio) (1.21.6)
Collecting paramiko
Downloading paramiko-2.10.4-py2.py3-none-any.whl (212 kB)
|████████████████████████████████| 212 kB 49.9 MB/s
Collecting markdown-it-py[linkify,plugins]
Downloading markdown_it_py-2.1.0-py3-none-any.whl (84 kB)
|████████████████████████████████| 84 kB 3.8 MB/s
Collecting python-multipart
Downloading python-multipart-0.0.5.tar.gz (32 kB)
Requirement already satisfied: pandas in /usr/local/lib/python3.7/dist-packages (from gradio) (1.3.5)
Collecting uvicorn
Downloading uvicorn-0.17.6-py3-none-any.whl (53 kB)
|████████████████████████████████| 53 kB 2.4 MB/s
Collecting analytics-python
Downloading analytics_python-1.4.0-py2.py3-none-any.whl (15 kB)
Requirement already satisfied: pillow in /usr/local/lib/python3.7/dist-packages (from gradio) (7.1.2)
Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.7/dist-packages (from aiohttp->gradio) (21.4.0)
Collecting aiosignal>=1.1.2
Downloading aiosignal-1.2.0-py3-none-any.whl (8.2 kB)
Requirement already satisfied: charset-normalizer<3.0,>=2.0 in /usr/local/lib/python3.7/dist-packages (from aiohttp->gradio) (2.0.12)
Collecting frozenlist>=1.1.1
Downloading frozenlist-1.3.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (144 kB)
|████████████████████████████████| 144 kB 70.9 MB/s
Collecting asynctest==0.13.0
Downloading asynctest-0.13.0-py3-none-any.whl (26 kB)
Collecting yarl<2.0,>=1.0
Downloading yarl-1.7.2-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (271 kB)
|████████████████████████████████| 271 kB 79.1 MB/s
Collecting multidict<7.0,>=4.5
Downloading multidict-6.0.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (94 kB)
|████████████████████████████████| 94 kB 3.1 MB/s
Requirement already satisfied: typing-extensions>=3.7.4 in /usr/local/lib/python3.7/dist-packages (from aiohttp->gradio) (4.2.0)
Collecting async-timeout<5.0,>=4.0.0a3
Downloading async_timeout-4.0.2-py3-none-any.whl (5.8 kB)
Requirement already satisfied: idna>=2.0 in /usr/local/lib/python3.7/dist-packages (from yarl<2.0,>=1.0->aiohttp->gradio) (2.10)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from analytics-python->gradio) (1.15.0)
Collecting monotonic>=1.5
Downloading monotonic-1.6-py2.py3-none-any.whl (8.2 kB)
Collecting backoff==1.10.0
Downloading backoff-1.10.0-py2.py3-none-any.whl (31 kB)
Requirement already satisfied: python-dateutil>2.1 in /usr/local/lib/python3.7/dist-packages (from analytics-python->gradio) (2.8.2)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->gradio) (2021.10.8)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->gradio) (1.24.3)
Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->gradio) (3.0.4)
Collecting pydantic!=1.7,!=1.7.1,!=1.7.2,!=1.7.3,!=1.8,!=1.8.1,<2.0.0,>=1.6.2
Downloading pydantic-1.9.0-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (10.9 MB)
|████████████████████████████████| 10.9 MB 81.4 MB/s
Collecting starlette==0.18.0
Downloading starlette-0.18.0-py3-none-any.whl (59 kB)
|████████████████████████████████| 59 kB 9.2 MB/s
Collecting anyio<4,>=3.0.0
Downloading anyio-3.5.0-py3-none-any.whl (79 kB)
|████████████████████████████████| 79 kB 9.5 MB/s
Collecting sniffio>=1.1
Downloading sniffio-1.2.0-py3-none-any.whl (10 kB)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.7/dist-packages (from Jinja2->gradio) (2.0.1)
Collecting mdurl~=0.1
Downloading mdurl-0.1.1-py3-none-any.whl (10 kB)
Collecting mdit-py-plugins
Downloading mdit_py_plugins-0.3.0-py3-none-any.whl (43 kB)
|████████████████████████████████| 43 kB 2.7 MB/s
Collecting linkify-it-py~=1.0
Downloading linkify_it_py-1.0.3-py3-none-any.whl (19 kB)
Collecting uc-micro-py
Downloading uc_micro_py-1.0.1-py3-none-any.whl (6.2 kB)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->gradio) (1.4.2)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->gradio) (3.0.8)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib->gradio) (0.11.0)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.7/dist-packages (from pandas->gradio) (2022.1)
Collecting bcrypt>=3.1.3
Downloading bcrypt-3.2.2-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (62 kB)
|████████████████████████████████| 62 kB 1.2 MB/s
Collecting cryptography>=2.5
Downloading cryptography-37.0.2-cp36-abi3-manylinux_2_24_x86_64.whl (4.0 MB)
|████████████████████████████████| 4.0 MB 51.0 MB/s
Collecting pynacl>=1.0.1
Downloading PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (856 kB)
|████████████████████████████████| 856 kB 77.3 MB/s
Requirement already satisfied: cffi>=1.1 in /usr/local/lib/python3.7/dist-packages (from bcrypt>=3.1.3->paramiko->gradio) (1.15.0)
Requirement already satisfied: pycparser in /usr/local/lib/python3.7/dist-packages (from cffi>=1.1->bcrypt>=3.1.3->paramiko->gradio) (2.21)
Collecting asgiref>=3.4.0
Downloading asgiref-3.5.1-py3-none-any.whl (22 kB)
Collecting h11>=0.8
Downloading h11-0.13.0-py3-none-any.whl (58 kB)
|████████████████████████████████| 58 kB 7.3 MB/s
Requirement already satisfied: click>=7.0 in /usr/local/lib/python3.7/dist-packages (from uvicorn->gradio) (7.1.2)
Building wheels for collected packages: ffmpy, python-multipart
Building wheel for ffmpy (setup.py) ... done
Created wheel for ffmpy: filename=ffmpy-0.3.0-py3-none-any.whl size=4712 sha256=33312190e6688d58b2b240b55e006abf43d734c87b2441b948a1d15ed71d544c
Stored in directory: /root/.cache/pip/wheels/13/e4/6c/e8059816e86796a597c6e6b0d4c880630f51a1fcfa0befd5e6
Building wheel for python-multipart (setup.py) ... done
Created wheel for python-multipart: filename=python_multipart-0.0.5-py3-none-any.whl size=31678 sha256=912a57d7429571fb2b51a5c2924e29d955736cd6334f05c578cba419ab22e5c8
Stored in directory: /root/.cache/pip/wheels/2c/41/7c/bfd1c180534ffdcc0972f78c5758f89881602175d48a8bcd2c
Successfully built ffmpy python-multipart
Installing collected packages: sniffio, mdurl, uc-micro-py, multidict, markdown-it-py, frozenlist, anyio, yarl, starlette, pynacl, pydantic, monotonic, mdit-py-plugins, linkify-it-py, h11, cryptography, bcrypt, backoff, asynctest, async-timeout, asgiref, aiosignal, uvicorn, python-multipart, pydub, pycryptodome, paramiko, orjson, ffmpy, fastapi, analytics-python, aiohttp, gradio
Successfully installed aiohttp-3.8.1 aiosignal-1.2.0 analytics-python-1.4.0 anyio-3.5.0 asgiref-3.5.1 async-timeout-4.0.2 asynctest-0.13.0 backoff-1.10.0 bcrypt-3.2.2 cryptography-37.0.2 fastapi-0.76.0 ffmpy-0.3.0 frozenlist-1.3.0 gradio-2.9.4 h11-0.13.0 linkify-it-py-1.0.3 markdown-it-py-2.1.0 mdit-py-plugins-0.3.0 mdurl-0.1.1 monotonic-1.6 multidict-6.0.2 orjson-3.6.8 paramiko-2.10.4 pycryptodome-3.14.1 pydantic-1.9.0 pydub-0.25.1 pynacl-1.5.0 python-multipart-0.0.5 sniffio-1.2.0 starlette-0.18.0 uc-micro-py-1.0.1 uvicorn-0.17.6 yarl-1.7.2
# Function to be passed in gradio for prediction
def gradio_single_test_prediction(array):
points = []
list_points = []
col = ['X1','Y1','X2','Y2','Area'] # Columns for contours
img_arr_org = array # Assign the input numpy array
img_arr_copy = np.copy(img_arr_org)
orig_im_h, orig_im_w, channels = img_arr_org.shape
img_arr_resize = cv2.resize(img_arr_copy, (128, 128)) # Resize the image
pp_img_arr = tf.keras.applications.mobilenet.preprocess_input(np.array(img_arr_resize, dtype=np.float32)) # Apply pre-process function
pp_img_arr_ = np.expand_dims(pp_img_arr, axis=0)
predict_vec = final_model_clf.predict(pp_img_arr_) # Predict Target
class_predicted = np.argmax(predict_vec)
if(class_predicted == 1):
pred_mask = final_model_unet.predict(pp_img_arr_) # Predict Bounding boxes
output = np.zeros(pp_img_arr.shape[:2], dtype=np.uint8)
for i in range(pred_mask[0].shape[1]):
for j in range(pred_mask[0].shape[0]):
if pred_mask[0][i][j] > THRESHOLD:
x1 = int(j * pp_img_arr.shape[1] / IMAGE_WIDTH)
y1 = int(i * pp_img_arr.shape[0] / IMAGE_HEIGHT)
x2 = int((j + 1) * pp_img_arr.shape[1] / IMAGE_WIDTH)
y2 = int((i + 1) * pp_img_arr.shape[0] / IMAGE_HEIGHT)
output[y1:y2, x1:x2] = 1
contours,_ = cv2.findContours(output, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE) # Finding all contours
for cnt in contours:
approx = cv2.approxPolyDP(cnt, 0.02*cv2.arcLength(cnt, True), True)
x, y, w, h = cv2.boundingRect(approx)
points.append(x)
points.append(y)
points.append(x+w)
points.append(y+h)
points.append(w*h)
list_points.append(list(points))
points=[]
df = pd.DataFrame(list_points, columns=col)
df.sort_values(by='Area', inplace=True, ascending=False) # Sorting the area of bounding boxes
list_points = []
df = df[:2] # Taking 2 sets of co-ordinates for 2 bounding boxes
if(df.shape[0] == 2):
x11, y11, x12, y12, _ = np.array(RATIO*np.array(df[:1])[0], dtype=np.int64)
x21, y21, x22, y22, _ = np.array(RATIO*np.array(df[1:])[0], dtype=np.int64)
img_arr_org = cv2.rectangle(img_arr_org, (x11,y11), (x12,y12), (220, 20, 60), 5) # Creating bounding box 1
img_arr_org = cv2.rectangle(img_arr_org, (x21,y21), (x22,y22), (220, 20, 60), 5) # Creating bounding box 2
label = 'Pneumonia'
elif(df.shape[0] == 1):
x11, y11, x12, y12, _ = np.array(RATIO*np.array(df[:1])[0], dtype=np.int64)
img_arr_org = cv2.rectangle(img_arr_org, (x11,y11), (x12,y12), (220, 20, 60), 5) # Creating bounding box 1
label = 'Pneumonia'
else:
label = 'Pneumonia'
else:
label = 'Non Pneumonia'
return img_arr_org, label, round(predict_vec[0][1],2) # Return Image, Label and Confidence
# Import gradio
import gradio as gr
# Launching gradio interface with required inputs and outputs
gr.Interface(fn = gradio_single_test_prediction,
inputs = gr.inputs.Image(shape=(1024, 1024), image_mode='RGB', type='numpy', label='Input Image'),
outputs = [gr.outputs.Image(label='Output Image'),
gr.outputs.Textbox(label='Predicted Label'),
gr.outputs.Textbox(label='Confidence')],
allow_flagging='never'
).launch()
Colab notebook detected. To show errors in colab notebook, set `debug=True` in `launch()` Running on public URL: https://13813.gradio.app This share link expires in 72 hours. For free permanent hosting, check out Spaces (https://huggingface.co/spaces)
(<fastapi.applications.FastAPI at 0x7fee40d8a110>, 'http://127.0.0.1:7860/', 'https://13813.gradio.app')
Going back to the main aim of this project which is to build an automated pneumonia detection system to locate the position of inflammation in an image, we would say that the presented model would achieve this to a certain extent, but the model can be improved further. Radiologists and technical experts must read and analyse high volumes of reports and present their findings. It's highly possible that they might miss some important information and record incorrect data in their findings which could be critical for the patient.
The current model can serve as the first step of screening through the DICOM images of the patients. After the model has categorized and predicted the bounding boxes, the healthcare specialist can prioritize patients for whom the category comes to be pneumatic and have big bounding boxes. The precise area must be further investigated by the healthcare specialists as the predicted area might be too large or not accurate enough. Also, there will be some false positives generated which can be confirmed by specialist. For those images predicted with very small bounding box, healthcare specialist can look at those with a lower priority as they are deemed to have a higher probability of no pneumonia.
One of the major challenges with this dataset is that there’s no defined shape of lung-opacity. Object detectors are good at identifying objects which have predefined shape.
Also there is no particular defined shape size of images provided, image quality is also varying.
Next big challenge was the computing power and GPU efficiency. This project is carried out on google Colab which did not provide with efficient GPU and hence modelling was restricted to only few epochs and finite set of data manipulation.
Due to compute restrictions, we've trained our models with images which has 2 bounding boxes.
Class-3 which has 'No Lung Opacity/Not Normal' is removed.